{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=6>1.4.深层神经网络</font>\n",
    "# 回顾\n",
    "1. 逻辑回归\n",
    "2. 参数初始化\n",
    "2. 前向传播、激活单元、损失函数、代价函数、后向传播\n",
    "3. 梯度下降-参数更新\n",
    "4. 向量化\n",
    "5. etc\n",
    "\n",
    "## 逻辑回归和浅层神经网络\n",
    "<img src='../images/7c1cc04132b946baec5487ba68242362.png'>\n",
    "\n",
    "## 神经网络层数的定义\n",
    "1. 从左到右\n",
    "3. 去除输入层\n",
    "4. 包含输出层\n",
    "4. 逻辑回归是一个一层的神经网络\n",
    "5. 层数=隐含层数+1\n",
    "<img src='../images/be71cf997759e4aeaa4be1123c6bb6ba.png'>\n",
    "\n",
    "## 案例\n",
    "1. $L:$表示层数，$L=4$\n",
    "2. $n^{[l]}:$表示第$l$层的节点(或特征)数，eg.$n^{[1]}=5$,$n^{[2]}=5$，$n^{[3]}=3$，$n^{[4]}$=$n^{[L]}=1$（输出单元为1）。而输入层，${n}^{[0]}={n}_{x}=3$。\n",
    "2. ${w}^{[l]}$来记作*l*层权重向量\n",
    "3. ${z}^{[l]}$来记作*l*层多元线性回归值\n",
    "3. ${g}^{[l]}$来记作*l*层激活函数\n",
    "3. ${a}^{[l]}$来记作*l*层激活值\n",
    "4. ...\n",
    "<img src='../images/9927bcb34e8e5bfe872937fccd693081.png'>\n",
    "\n",
    "## 符号\n",
    "<font size=3>\n",
    "<ul>\n",
    "<li>上标 $^{(i)}$ 代表第 $i$ 个训练样本</li>\n",
    "<li>上标 $^{[l]}$ 代表第 $l$ 层</li>\n",
    "<li>$m$ 数据集的样本数</li>\n",
    "<li>下标 $_x$ 输入数据</li>\n",
    "<li>下标 $_y$ 输出数据</li>\n",
    "<li>$n_x$ 输入大小</li>\n",
    "<li>$n_y$ 输出大小 (或者类别数)</li>\n",
    "<li>$n_h^{[l]}$ 第 $l$ 层的隐藏单元数</li>\n",
    "<li>$L$ 神经网络的层数</li>\n",
    "<li>$n_x = n^{[0]}$</li>\n",
    "<li>$n_y = n^{[L]}$</li>\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 前向传播和反向传播\n",
    "## 前向传播\n",
    "### 单个样本\n",
    "$z^{[l]}=W^{[l]}\\cdot{a}^{[l-1]}+{b}^{[l]}$\n",
    "\n",
    "$a^{[l]}=g^{[l]}\\left(z^{[l]}\\right)$\n",
    "### 全体样本集-向量化\n",
    "${Z}^{[l]}={W}^{[l]}\\cdot {A}^{[l-1]}+{b}^{[l]}$\n",
    "\n",
    "${A}^{[l]}={g}^{[l]}({Z}^{[l]})$\n",
    "## 反向传播\n",
    "### 单个样本\n",
    "1. 输入：$da^{[l]}$\n",
    "2. 输出：$da^{[l-1]}$,$dw^{[l]}$,$db^{[l]}$\n",
    "\n",
    "反向传播的步骤可以写成：\n",
    "\n",
    "（1）$dz^{[l]}=d{a}^{[l]}*{g^{[l]}}'({z}^{[l]})$\n",
    "\n",
    "（2）$dw^{[l]}=d{z}^{[l]}\\cdot{a}^{[l-1]}~$\n",
    "\n",
    "（3）$db^{[l]}=d{z}^{[l]}~~$\n",
    "\n",
    "（4）$da^{[l-1]}=w^{\\left [ l \\right ] T}\\cdot{dz}^{[l]}$\n",
    "\n",
    "（5）$dz^{[l]}=w^{[l+1]T}dz^{[l+1]}\\cdot{g^{[l]}}'({z}^{[l]})$\n",
    "\n",
    "式子（5）由式子（4）带入式子（1）得到，前四个式子就可实现反向函数。\n",
    "### 全体样本集-向量化\n",
    "（6）$d{Z^{[l]}}=d{A^{[l]}}*{g^{\\left[ l \\right]}}'\\left({Z^{[l]}} \\right)~~$\n",
    "\n",
    "（7）$d{W^{[l]}}=\\frac{1}{m}\\text{}d{Z^{[l]}}\\cdot {A^{\\left[ l-1 \\right]T}}$\n",
    "\n",
    "（8）$d{b^{[l]}}=\\frac{1}{m}\\text{ }np.sum(d{z^{[l]}},axis=1,keepdims=True)$\n",
    "\n",
    "（9）$d{A^{[l-1]}}={W^{\\left[ l \\right]T}}.d{Z^{[l]}}$\n",
    "\n",
    "<img src='../images/53a5b4c71c0facfc8145af3b534f8583.png'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 矩阵的维数\n",
    "## 单个样本\n",
    "$w$的维度是（下一层的维数，前一层的维数），即${w^{[l]}}$: (${n^{[l]}}$,${n^{[l-1]}}$)；\n",
    "\n",
    "$b$的维度是（下一层的维数，1），即:${b^{[l]}}$ : (${n^{[l]}},1)$；\n",
    "\n",
    "${z^{[l]}}$,${a^{[l]}}$: $({n^{[l]}},1)$;\n",
    "\n",
    "${dw^{[l]}}$和${w^{[l]}}$维度相同，${db^{[l]}}$和${b^{[l]}}$维度相同，且$w$和$b$向量化维度不变，但$z$,$a$以及$x$的维度会向量化后发生变化。\n",
    "## 样本集向量化\n",
    "$Z^{[l]}$可以看成由每一个单独的$z^{[l](i)}$叠加而得到，${Z}^{[l]}=({z^{[l](1)}}，{z^{[l](2)}}，{z^{[l](3)}}，\\dots，{z^{[l](m)}})$，\n",
    "\n",
    "$m$为训练集大小，所以$Z^{[l]}$的维度不再是$({n^{[l]}},1)$，而是$({n^{[l]}},m)$。\n",
    "\n",
    "$A^{[l]}$：$(n^{[l]},m)$，$A^{[0]} = X =(n^{[l]},m)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 参数VS超参数\n",
    "1. 参数:$w,b$\n",
    "2. 超参数:\n",
    "    * **learning rate** $a$（学习率）\n",
    "    * **iterations**(梯度下降法循环的数量)\n",
    "    * $L$（神经网咯的层数）、${n^{[l]}}$（隐藏层单元数目）\n",
    "    * **choice of activation function**（激活函数的选择）\n",
    "\n",
    "超参数实际上控制了最后的参数$W$和$b$的值，所以它们被称作超参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 创建深层神经网络\n",
    "## 库包和环境配置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif'] = ['SimHei'] #指定默认字体,用来正常显示中文标签\n",
    "plt.rcParams['axes.unicode_minus'] = False #解决保存图像是负号'-'显示为方块的问题\n",
    "%matplotlib inline\n",
    "np.random.seed(1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 业务全景\n",
    "<img src=\"images/final outline.png\" style=\"width:800px;height:500px;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参数初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def initialize_parameters_deep(layer_dims):\n",
    "    \"\"\"\n",
    "    Arguments:\n",
    "    layer_dims -- (list) 从0到L，每一层的特征(节点)数\n",
    "    \n",
    "    Returns:\n",
    "    parameters -- (dictionary) 每一层参数值 \"W1\", \"b1\", ..., \"WL\", \"bL\":\n",
    "                    Wl -- 第l层的权重矩阵，shape= (layer_dims[l], layer_dims[l-1])\n",
    "                    bl -- 第l层的截距向量，shape (layer_dims[l], 1)\n",
    "    \"\"\"   \n",
    "    np.random.seed(1)\n",
    "    parameters = {}\n",
    "    L = len(layer_dims)# 层数+1\n",
    "    for l in range(1, L):\n",
    "        parameters['W' + str(l)] = np.random.randn(layer_dims[l],layer_dims[l-1])/ np.sqrt(layer_dims[l-1])\n",
    "        parameters['b' + str(l)] = np.zeros((layer_dims[l],1))\n",
    "    return parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'W1': array([[ 0.93781623, -0.35319773, -0.3049401 ],\n",
       "        [-0.61947872,  0.49964333, -1.32879399]]), 'b1': array([[0.],\n",
       "        [0.]]), 'W2': array([[ 1.23376823, -0.53825456]]), 'b2': array([[0.]])}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "initialize_parameters_deep([3,2,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 前向传播\n",
    "### 多元线性函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_forward(A, W, b):\n",
    "    \"\"\"\n",
    "    计算某层的多元线性值\n",
    "    Arguments:\n",
    "    A -- 前一层的激活值 (或者输入值)，形状=(前一层的节点数, 样本数目)\n",
    "    W -- 当前层权重矩阵，形状= (当前层节点数, 前一层的节点数)\n",
    "    b -- 当前层截距向量，形状= (当前层节点数, 1)\n",
    "\n",
    "    Returns:\n",
    "    Z -- 多元线性回归值，即当前层激活函数的输入值，又叫预激活值\n",
    "    cache -- （dictionary） 存储 \"A\", \"W\" and \"b\" 等值， 以便后面后向传播使用\n",
    "    \"\"\"\n",
    "    Z = np.dot(W,A) + b\n",
    "    cache = (A, W, b)\n",
    "    return Z, cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 激活函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(Z):  \n",
    "    A = 1/(1+np.exp(-Z))\n",
    "    cache = Z    \n",
    "    return A, cache\n",
    "\n",
    "def relu(Z):    \n",
    "    A = np.maximum(0,Z)    \n",
    "    cache = Z \n",
    "    return A, cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 前向传播函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_activation_forward(A_prev, W, b, activation):\n",
    "    \"\"\"\n",
    "    执行第l层的前向传播\n",
    "    Arguments:\n",
    "    A_prev -- 前一层的激活值 (或者输入值)，形状=(前一层的节点数, 样本数目)\n",
    "    W -- 当前层权重矩阵，形状= (当前层节点数, 前一层的节点数)\n",
    "    b -- 当前层截距向量，形状= (当前层节点数, 1)\n",
    "    activation -- 当前层的激活函数类型: \"sigmoid\" or \"relu\"\n",
    "\n",
    "    Returns:\n",
    "    A -- 多元线性回归值，即当前层激活函数的输入值，又叫预激活值\n",
    "    cache -- （dictionary） 存储 \"linear_cache\" and \"activation_cache\"等值， 以便后面后向传播使用\n",
    "    \"\"\"    \n",
    "    if activation == \"sigmoid\":\n",
    "        Z, linear_cache = linear_forward(A_prev,W,b)\n",
    "        A, activation_cache = sigmoid(Z)  \n",
    "    elif activation == \"relu\":\n",
    "        Z, linear_cache = linear_forward(A_prev,W,b)\n",
    "        A, activation_cache = relu(Z)\n",
    "    cache = (linear_cache, activation_cache)\n",
    "    return A, cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### $L$层前向传播模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def L_model_forward(X, parameters):\n",
    "    \"\"\"\n",
    "    神经网络前向传播\n",
    "    \n",
    "    Arguments:\n",
    "    X -- 特征数据集, 形状=(特征数,样本数)\n",
    "    parameters -- 初始化参数：初始化函数的输出值\n",
    "    \n",
    "    Returns:\n",
    "    AL -- 预测值：输出层的激活值\n",
    "    caches -- （list） 存储:每一层的多元线性函数和激活函数的返回值\n",
    "    \"\"\"\n",
    "    caches = []\n",
    "    A = X\n",
    "    L = len(parameters) // 2\n",
    "    for l in range(1, L):\n",
    "        A_prev = A \n",
    "        A, cache = linear_activation_forward(A_prev,parameters['W' + str(l)],parameters['b' + str(l)],activation = \"relu\")\n",
    "        caches.append(cache)\n",
    "    AL, cache = linear_activation_forward(A,parameters['W' + str(L)],parameters['b' + str(L)],activation = \"sigmoid\")\n",
    "    caches.append(cache)            \n",
    "    return AL, caches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 成本函数\n",
    "成本值$J$, 计算公式: $$-\\frac{1}{m} \\sum\\limits_{i = 1}^{m} (y^{(i)}\\log\\left(a^{[L] (i)}\\right) + (1-y^{(i)})\\log\\left(1- a^{[L](i)}\\right)) \\tag{7}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_cost(AL, Y):\n",
    "    \"\"\"\n",
    "    计算模型成本值.\n",
    "    Arguments:\n",
    "    AL -- 标签预测值, 形状=(标签维数=1, 样本数=m)\n",
    "    Y -- 真实标签值（0 or 1）, 形状=(标签维数=1, 样本数=m)\n",
    "    Returns:\n",
    "    cost -- 交叉熵\n",
    "    \"\"\"    \n",
    "    m = Y.shape[1]\n",
    "    cost = -1 / m * np.sum(Y * np.log(AL) + (1-Y) * np.log(1-AL),axis=1,keepdims=True)    \n",
    "    cost = np.squeeze(cost) #转为标量值\n",
    "    return cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 后向传播\n",
    "<img src=\"images/backprop_kiank.png\" style=\"width:650px;height:250px;\">\n",
    "\n",
    "<img src=\"images/linearback_kiank.png\" style=\"width:250px;height:300px;\">\n",
    "$$ dW^{[l]} = \\frac{\\partial \\mathcal{L} }{\\partial W^{[l]}} = \\frac{1}{m} dZ^{[l]} A^{[l-1] T}$$\n",
    "$$ db^{[l]} = \\frac{\\partial \\mathcal{L} }{\\partial b^{[l]}} = \\frac{1}{m} \\sum_{i = 1}^{m} dZ^{[l](i)}$$\n",
    "$$ dA^{[l-1]} = \\frac{\\partial \\mathcal{L} }{\\partial A^{[l-1]}} = W^{[l] T} dZ^{[l]}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多元线性函数微分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_backward(dZ, cache):\n",
    "    \"\"\"\n",
    "    第l层多元线性函数的微分\n",
    "    Arguments:\n",
    "    dZ -- 第l层多元线性值的微分\n",
    "    cache -- 第l层，前向传播的缓存值-(A_prev, W, b)\n",
    "    Returns:\n",
    "    dA_prev -- 第l-1层的激活值得微分, 形状同A_prev；\n",
    "    dW -- 当前层权重矩阵的微分, 形状同W\n",
    "    db -- 当前层截距向量的微分, 形状同b\n",
    "    \"\"\"\n",
    "    A_prev, W, b = cache\n",
    "    m = A_prev.shape[1]\n",
    "    dW = 1 / m * np.dot(dZ ,A_prev.T)\n",
    "    db = 1 / m * np.sum(dZ,axis = 1 ,keepdims=True)\n",
    "    dA_prev = np.dot(W.T,dZ) \n",
    "    return dA_prev, dW, db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 激活函数微分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def relu_backward(dA, cache):\n",
    "    Z = cache\n",
    "    dZ = np.array(dA, copy=True)\n",
    "    dZ[Z <= 0] = 0       \n",
    "    return dZ\n",
    "def sigmoid_backward(dA, cache):\n",
    "    Z = cache    \n",
    "    s = 1/(1+np.exp(-Z))\n",
    "    dZ = dA * s * (1-s)    \n",
    "    return dZ"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 后向传播函数微分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_activation_backward(dA, cache, activation):\n",
    "    \"\"\"\n",
    "    后向传播    \n",
    "    Arguments:\n",
    "    dA -- 当前层激活值的微分\n",
    "    cache -- 当前层前向传播的缓存值： (linear_cache, activation_cache)\n",
    "    activation --  当前层的激活函数类型: \"sigmoid\" or \"relu\"    \n",
    "    Returns:\n",
    "    dA_prev -- 第l-1层的激活值得微分, 形状同A_prev；\n",
    "    dW -- 当前层权重矩阵的微分, 形状同W\n",
    "    db -- 当前层截距向量的微分, 形状同b\n",
    "    \"\"\"\n",
    "    linear_cache, activation_cache = cache    \n",
    "    if activation == \"relu\":\n",
    "        dZ = relu_backward(dA, activation_cache)\n",
    "        dA_prev, dW, db = linear_backward(dZ, linear_cache)\n",
    "    elif activation == \"sigmoid\":\n",
    "        dZ = sigmoid_backward(dA, activation_cache)\n",
    "        dA_prev, dW, db = linear_backward(dZ, linear_cache)\n",
    "    return dA_prev, dW, db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### $L$层后向传播模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def L_model_backward(AL, Y, caches):\n",
    "    \"\"\"\n",
    "    神经网络后向传播    \n",
    "    Arguments:\n",
    "    AL -- 标签预测值, 形状=(标签维数=1, 样本数=m)，前向传播的输出值\n",
    "    Y -- 真实标签值（0 or 1）, 形状=(标签维数=1, 样本数=m)\n",
    "    caches -- (list)存储:每一层的前向传播中多元线性函数和激活函数的返回值\n",
    "    \n",
    "    Returns:\n",
    "    grads -- （dictionary）所有参数的梯度下降值，即所有参数的微分值\n",
    "    \"\"\"\n",
    "    grads = {}\n",
    "    L = len(caches)\n",
    "    m = AL.shape[1]\n",
    "    Y = Y.reshape(AL.shape)\n",
    "    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))\n",
    "    current_cache = caches[L-1]\n",
    "    grads[\"dA\" + str(L)], grads[\"dW\" + str(L)], grads[\"db\" + str(L)] =\\\n",
    "    linear_activation_backward(dAL, current_cache, activation = \"sigmoid\")\n",
    "    for l in reversed(range(L - 1)):\n",
    "        current_cache = caches[l]\n",
    "        dA_prev_temp, dW_temp, db_temp = \\\n",
    "        linear_activation_backward(grads[\"dA\" + str(l+2)], current_cache, activation = \"relu\")\n",
    "        grads[\"dA\" + str(l + 1)] = dA_prev_temp\n",
    "        grads[\"dW\" + str(l + 1)] = dW_temp\n",
    "        grads[\"db\" + str(l + 1)] = db_temp\n",
    "    return grads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 梯度下降-更新参数\n",
    "$$ W^{[l]} := W^{[l]} - \\alpha \\text{ } dW^{[l]}$$\n",
    "$$ b^{[l]} := b^{[l]} - \\alpha \\text{ } db^{[l]}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_parameters(parameters, grads, learning_rate):\n",
    "    \"\"\"\n",
    "    梯度下降-更新参数    \n",
    "    Arguments:\n",
    "    parameters -- 更新前的参数\n",
    "    grads -- 后向传播的输出-所有参数的梯度下降值    \n",
    "    Returns:\n",
    "    parameters -- 更新后的参数\n",
    "    \"\"\"    \n",
    "    L = len(parameters) // 2\n",
    "    for l in range(L):\n",
    "        parameters[\"W\" + str(l+1)] =  parameters[\"W\" + str(l+1)] - learning_rate * grads[\"dW\" + str(l + 1)]\n",
    "        parameters[\"b\" + str(l+1)] = parameters[\"b\" + str(l+1)] - learning_rate * grads[\"db\" + str(l + 1)]\n",
    "    return parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 应用深层神经网络对图片进行分类\n",
    "## 数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import h5py\n",
    "def load_data():\n",
    "    train_dataset = h5py.File('datasets/train_catvnoncat.h5', \"r\")\n",
    "    train_set_x_orig = np.array(train_dataset[\"train_set_x\"][:]) # 训练集的特征\n",
    "    train_set_y_orig = np.array(train_dataset[\"train_set_y\"][:]) # 训练集的标签\n",
    "\n",
    "    test_dataset = h5py.File('datasets/test_catvnoncat.h5', \"r\")\n",
    "    test_set_x_orig = np.array(test_dataset[\"test_set_x\"][:]) # 测试集的特征\n",
    "    test_set_y_orig = np.array(test_dataset[\"test_set_y\"][:]) # 测试集的标签\n",
    "\n",
    "    classes = np.array(test_dataset[\"list_classes\"][:])# 分类类型\n",
    "    \n",
    "    train_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0]))\n",
    "    test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.shape[0]))\n",
    "    \n",
    "    return train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x_orig, train_y, test_x_orig, test_y, classes = load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y = 1. It's a cat picture.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMYAAADDCAYAAADZVm7uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztvWmwJtd5Hvac7v72u8+KGWCAATAAAS4AKJgCRFEEKZKSHNtxGFmSY8VJOSlWUoz/+I/liuwfKVUqpXKp4nIVXWGKTLm8JCUpjkxZiSLLJgNKJkhxFwiQBAjMAJjt3pm7ffvSffLjfU+/z4fbmLkzxAxxwfNWTd2e83WfpZfz7s/rvPeIFCnSPCU/6glEivRWpPhhRIpUQfHDiBSpguKHESlSBcUPI1KkCoofRqRIFRQ/jEiRKih+GJEiVdBNfxjOuc84577knPv1N3NCkSK9FSi7mYuccx8HkHrvn3TOfdY5d8Z7/0LVuUnifJKG6/TvXF/Y26jO+GYjLZua9Zr8xJ56Og7tRWFtTjvnttEkf93Adh4NjTwveJZ7FxaaKgIHuCnhvsN8ad5JIntTs9ko29YOHQEAZKk9Hk+9FvkUADDoXbWB8tmescO6nOO1yD11ifWdpLY/+qLQedm9L+9PxVocrG+vc+D+Z/nEpljRd5Hn2o/1naTyrPm5FchpPj40ljTL9z6IXK8P97jXn2I0zise5l66qQ8DwFMAfluP/wjATwMoPwzn3CcAfEImBaysycQyHS2z+4I0dfobzVcX/ODdS2XTO04fBwBMRxM6z27WZCQvS28wLtvq+jENRnYHv3t2EwDga/Yi1mp1Glpu5tbOgCcJAEj4A9GXjaZQvpT0PNGoW9+zib68uV3U6cg8HnrHfWXbf/ar/w0AYGX5sPVd2Lp3dy4CAL75J//Cfu9fmZs/AGS1TKc/LNsStyrz6ljfrU6rPB5PegCAhdaaXaMveegPAKZTuc8pbF7T7hUbe0H639p5pWzb3pW+m4urZdtwdwcAUC/s3raWTgAA+mP70Maz3fK4VpNnnQ/suV7t6fOg9ff1XWg22wCA3//jV7FfutkPowPgvB5vAngv/+i9/zSATwNAkjrf03csLD2Z+2Z156FdTd9DPHTKXt53nL4LAPDd575fto2n9oKNRnITZ1PruVaTjloteznrNRm82aKlp/alTpRTrC3Zy1LTL7qgXamrH2h/Yi/GXr4DTCb2oaa649aaNnanLR/v6pLNcePVbwAAhpvLZVujYfMp9Gus0a4/S/V34kY1/fiTpG1t9QVtK5swmdqHk2Uyj4SlbO2SN6Wk3N1sXmgeKw/7wy0AwFbXrqnXDwEA8oG1NdIOACD1dF6xpf0dKtumO83y+PJV+UgauhYAuLKzDgAYTewFCFwogazJF/uPC7xZHaMHuyMLP0Q/kSK9JelmOcbXIOLTMwAeAfC9NzzTAz4vDwHMs7sk2SvyBVE2pd/C0XRq7HU6pZ1Z2+dVENn9hwPjLEF12O3bDtWxzRoLDRUbOsatknSvIrTQULm1btymqzvhlPQTT8ur692u1WwfyXSNLEkilx18PKR5T+yaYiZrTcY7ZVvqhTMlmXGHlnKZrGkLzJQ77g76ZdulDRN37jp8Usb29mqEjXY47pZtY92ZD68YlxjT79td2cE7xOm2d0SMraV0H9sics2SxbJtMJTzXLZl60usn1ZLzmW9pFOXdffHdo3Xm19vCLdxyf7375v9MH4PwBedcycA/AKAJ26yn0iR3pJ0UyKQ934XooA/A+BD3vuda18RKdLBopvlGPDeb8EsU/u8SP+yAUqlDldhRCMrInIVHwoSU9gUWlPT5nBs4sfli6L1v7ZhymU+k98bTTL1zuw4V4U+o8GDOZPFvmBxXCST8qKy8zlzLa2hrsaANLPGmlp6BtubZdv3vvMcAGBhwZTLNlmOak7m2L/yWtnW0nnUVk2UaqgI5cg82mqKsp/npswuNGycJJX24cjEolc3zsq8eTF6K5baZjkcDM1y1GmJiHT+8g/Kts3uCABw97ETZZt3MreCrFIFdA0Duyd5as+9EayIibW11JjS6Nu6JoWIe42GrNlVvWRvQFFpjhSpgm6aY9wUlX6ia3+5YWNu1Hhnla9+Rn6D3YEp4i9fEq7w6vqobOvq74W3nWWxLX0WI+toltgeP5vKuTXe1dU0mZHSGDgK76JVnMUR/wjcypPzo9C26diMATubIpnyfeJxskzWtbpofR/xsv42mb2DD2UysXtSKKeoORvv5JpxmWkuXLbVNE7QVENDu267MfSeJs64caNp/Sy0xdTa7xsXySEKeV6Y4j8aSN9OzbaA+baGxEXWdy+Xx2sLovAzl1lUhbxzJ3Gwsaylps/lBhhG5BiRIlVR/DAiRaqg2ytKVVAZc8OOVrU/b+4au3/uJVHEvv6ihWqcvWwseaCe7/lgoRDPQ33r3zH5QKYkfoxUlCKpqVSgU47nUXGJfS1B3JkXufaKXxmJaWkQycj7nqUa40M3hcWzjhoOlhdMdIEqy9PCPO1buyK6LNQpLsqpv8CZb2O0azFXzdGGzMfVyrYTiytyCawt+BBmQzNIOuqzmIlI025a+Iefyb0NfggACI7qnKIKpmNpLED3xNlxMLp48pY3VcybFTYHp9eHOLMbkKQix4gUqYpuC8cIXOGajkfa6YMJ9xvfNXPdt5+T4+2x7e5llCVRlYLFbcHTTDr8/K6vh+yQr9ppwpqmtNOFCE/mRnPzKHeuvZ3zHMN8Erph/PuJI7IrprSDd1aEewwLM0i0cuEitboptoWX35tti8Ny3nbj2kw8xxxX1E5lvHFBUbH6kLpd41BZShGwiSj8Ic4MANJUuEhO/VisHBlD1CCRkVGkUS/od5Ea+N4HBTvNbK3BTD0uDRvRXBsp0g9F8cOIFKmCbosoZWJASJzhBKO95wcJYi45J4hjLD7NiVIVmUM+/LJX/WalmEWprEKUCsreXILV6wcBBQz6vSHbcij/8VVt1HeZdENOm5wuck4CHFs0zEJL2o7fcbxsKzS3Ykhh5RtXJWy/3TaleK1tuRdOjQBD8vM4DdCckgjkocquszDvpeWjNl+VfCYj86GUyVi0llTFJU4MKzT03dN9XGiY2DjV6AQ2ThSFiFB0mgVr5sHncuvDziNFelvTj8Rc6/3eHZwp5MCQ3oaabuGUh4RJfm3OE/Z1VvqD2XNeIacdTHeh1FUp5OzRvka66xyb4BigvZMMHCMkVQHA4qLsfrXMtr/BsFce1+tqrqQkqXpNFNtiZObsJJWdvts1s3ZbWWIzNaWZFd9R+24AwCQzJb6WBW+5heKHddXGNt5sZCHfSaomXtrVXSFrmEuHVcU/cfawa3UZZzy2e9ckr7pXLz9nggYJoEch9KkaGgKj8wW/PdemyDEiRaqg+GFEilRBt16UciBb/V7lsyTWqat+dsHTTI3kjAjX1Ehuaqh4UidFu6XSSU6BhSmx+3DMGXUhXLxDueMmIhDChvdzf4F5j34IhFyloL13vesMAOA9j5ws2+6/TxTozoKFZ29d+Hx5/OUvfBkA0L9oootThZW9z9DAvEbLRK7lZQnu8xMKZJwS8EOiOdh1ApVQMW1C+dSFD74Gu3Q8MpGtHrIGKRNwfV3ACDokAtWXRPFnz79T7zwr1/yCNBoh7Nx+nY408HJq4fKlQt8JxoX984HIMSJFqqD4YUSKVEH7EqWcc8cA/K73/gPOuRqAfwVgDcBnvPefvd71wZoTrEyzWZX/gc5Xy1G3R4Flek3KqDfkbGgoT7/nkFlOji6LNeX+4xa0dt89IqZsERjCsz+4WB6/tiHiwKywPePwmogSjz1s4s4HPvgeAEB7xcSdYG3JKd/Ckby3pOeurFLboswjTclylEj4S7b4zrLt7tMfL48XmrKuz//+02UbNPMuzSg4MNznGeFB7Up+RJJZ3kIBsjapKFpMKfSmvjfQM1f/hM9NvJqYIQtZuH8U6Neu6XworCcAVTD2VU0BKeqE9xXCQADAKZbVeGJZjV5Ba2Y1C3VJM+nTqQh7IwkZ1+UYzrlVAP8UgiUFAH8bwNe89+8H8IsuCITz13zCOfdV59xXY4m/SAeR9sMxcgC/DOBf6/+fAvBrevw0gMcBfJ4veD3gWmAKZYi5mztX2zg8Wwcm5TrEtFWrY0CqF+1MTYnrb4gCedeq7YiLbfm+7zxxT9n2yJm7yuPdrii0r1yyHXy9K33OhuZXuPDyOQDAB3/OPM2dZd07Ept4o26gYfWG7Gb1lrX5XELDC/LKFgNRrqfdPyvbks77bY79jl5Djy+XOW5sWRj4Zl9RF8nfc9dhycVuUobeLKedeSgh6FnGAYPCClKySHhVvhOwd944QT6VXX02M04QYsznPNZqLJiSl3+oHvuE/Es1yvmeavh6kdjzKNRL7mq2rjQT736tEXxXbyJ8jiKC8Iv7ehTCYxWXRYp0oOlmlO+IQhjpbU8348cIKIS/C0EhfOZaJztwQF5AH6ffK/IRmi0Nedjh0IFK70Z5NFLYnDxnXFS5/ty6iRff/FdfkispkX5txRS2R+8Xm/dDJy2uf7UjatSlHRvvuedF5Dhxx3fLtjvv1ay1NVPSh/2XyuOlxYd0bAusS7NgY6e8jq7sNbPhhbJtkF0qj1949usAAOfN11BARBaGsDmxtNdv4p2Easym1SGRAUndwfqeeRFFHWXRpTXZG/3QxLB2Yj6E2VjAC6Yk2gZEQJ8ZXE+APUoJA3fj6jYAYDw0cbZGINwL6oyqZRTKo76mrGbvTLcn/Zx/TeY1JMCJ69HNfBj/FMD/7Zz7AICHAXz5JvqIFOktTfv+MLz3T+nfc865j0K4xj/wQQu7FrlQu6IiNLwi0G8WMuDmcGhDGwWlEReZqnY+nZr50Ktutr5tHV0ZBq5l117atUzB3Z7s5i++ZjtYvSY7XI9CsUOA3re/art/vSkcYSU3BRCFmRlzzcsO2W0AUNNSBcPRetk22pK+d6/Y7vfKlT8pj//sSxIod+Ze4wSFwvIvEMTNoCtcpN+1OSyuyHmdBVvfjOpadIeyqy5yvKBXLpLbDu58yF83jrE7tB25p571Tt06miTCmRMO5ssVp5YMCSsdvSfEoa70KehR62ccNaaOJJGxtyhg8txl4epLHfXmV+AkvxHdVEiI9/4CbhSFMFKkA0RRcY4UqYJueRChhwUNhr/zSIQBKMHaSv9FxXlv5DAMQANcG2Q6k+t3R1RlSPscUkcjyh7bUPFjzGWzMJhfAICTd0q22tqaKe6bipE7GFlOgCfRbqMp4+xsmiI9UVFrNiHI+4n0uTQxH8Aff+XF8nh3IAr2oTUuQCNiVWthpWxrqrQ0nVFJA72lk7HNa5dKlvWGorD2BibGHFKvfI3yLSZOxiuapkgPqALSSMXhxQYhOipG7mhguSVO92ZHz7/f0ypL5PlebVvfiaIo5lMTSUNIRELi1/KSzHEyufa7U0WRY0SKVEG3JYPPQof2wseEnYLTcYNVrSq0xVflWMNC2nnnCXsM4bahpgjZ3TF5VemakFs+GNoOFfBuGXP2jiW5dd1120W/8R0Jq86aFiXTHdiuP558CwCwSd7pgXqDW1Tf7uffId70dzxoOdQjKtnV0zim754z0+y9x2WOOSn7HQVKazSMYzjFjZ2Q6TLL7Zq1jniLdwY2x25f1rjSMGU/GUs/eW4aMIf8r7UVfZxqEAbInUbLuEwo+DMc2n0MZdwyitc6tkzWgPI5cBEhOR5PqaCnpu6NZjJXxjC+HkWOESlSBcUPI1KkCrqt8Dml8lOFhUBtef76H+n0N6q8WVFEPIhXzQVr7A32hlAPCTlwW0UoqkCAvnrTM/K0njsnYtPDFOZcV/zcZ75nPokB9R3s6ItUtXVRQ6zvpEI2JzQYcallPomVtk2or9D5VLQWIzU01IaceSeKdI0g9otwgwjjNnHsGQ+FY0y0OaRpk8XQPNup+hJmdRMbl6hAzUTDzodDKhMwlfm4RQvVH2uJghk960OLModdupbXmmYyJr8LvaH088plMySsLor1YVXBI9JYOCZSpB+O4ocRKVIF3SZcqXkEQl8REjInXVWJXOVvnBNAoAPXGJ2rMJVXzOWE2HGh7Lkgtht+zmnsTc0AXN82keTBYyJWve8+CwkZE5R9qNZ0aMEsNZc2xSJ07hULS1lQa1JBuEpzINOaF7GwYKKW00VMKIiwmwcfEeWEtMTSlWRm5akTftXOrvhYZnRTBipqTVOzArU1n4Oh+l1q6/aF3B8/tOzI/lDWWqdMv0TLBGxT2MpYfT8ZAYtdsWVhRStJlRmBAJYXJfvhjtzW9Rfe+xgAoKUWsqe//cfYL0WOESlSBf1olG+GytENh/Q25EWFv6NCEZ8vj1rBZrRpNLmOy5OcKA2FceEi7YkWlmGuNNDd+GLXtr+Z5hjfMbPFrC1QOLl6hi/17bY/+4oopB32pUxk9xw7w5RtUZGY8e5A52r91OuiaNYoo84lgfuZN9ypcs3RB55KB3j1NSRgjBvxOwympNg3ZG75xLzYKVV/dRPpZ0ZoiQNFNEwLTg2QNRxeNizdqUYdXN0xX0rBW/hMFO2csqqf/Jn/FACwu7NRti0sHwEAXL54Xte2/9c9coxIkSoofhiRIlXQbaqoVP2XqSqwcC7H7PXiGPA6Ff6N1e+5SIDKikkURqKdNitQ8Hjskfa5TaElDRXZrl6wnICmM/EjhEes9y34bVMrEr3/jIk7mYpkXRJT6g1T2EPFpsGO+RVWm2rEaJnyOU1EdDm3fqVsO31CRC5WuAsSbUKAYuHt94WWGBW2xhT8qAF8SW5rGY3sdbq6KyKiZX0AhYa/pG07z6mY2mlwfogCXa/SHCnQc6Z+pedeIuCHXDIzr26aH6Pe0Kw+FeF6dN+vR5FjRIpUQdflGM65ZQD/B4AUQB8CpfNPIGmtf+C9/41rduDNBFpV3CVg0XLBj1lFaq5dydD/b2BzfR0x/H4SMgbnGAwBu+luVWR8zV4KBoIBJbB31eucZhTGvmOKbRaqp1KwXUfzl5c6xhEWFuU8R8XlD5ESv6pe8JTG7vd1HGcKcF4TRbzbt511OBAveNI2ZX5G5tOGmkhrme3gtUTbCF92p6swO1TxdosquM600w4VpVmC/F5QnTwfxiaonGDWr1F24JQKQBQzuWcXiDt85cV/L+eRbb6siKtQQLtvMsf4GwB+y3v/MQCXAPwKgNR7/ySAe51zZ/Y9WqRIB4T2gyv1KfrvEQC/CuB/1v//EST3+wW+xjn3CQCfkP+8GdOMFOn20r6Vb+fckwBWAZzFPODae19/7hwSIRU+q4rhCjDyzToF+vUq4uarSuyx9FThIynLD1R8nTyXOWREXMtCsBfOZ4fzGpTdz/VHGWU1RUG866jZ3/NtLVLfoSw7HZvR9x4+bl7u/oaIGBmJUlkhYleNjAGjmWbHEarg5hWx8087Jiq1SNxpNY7sWUM+EyW/QZ7okWYXMgBCw5tM1lLvfJ9gelaWNHgwtbXOZsFvwiKy3kdCGuQyAdOpGDd2B2bY6A5kbF/h7wrBm0XxJudjOOfWAPxjAH8LEXAt0o8B7Uf5rgP4HQB/T6FzAuDaMxDAte/td7DKnO9ghqWPmcHQ7LS9kDpzFVOrZz83row9//f1/wncZe5rL7mR23PJlJDbB4p2Xq/ZefUGwcKsikl2ac04Rqa18JpN4whZWfDGFMlV4ij3HRYFe9o3c25Nldd23ZTUTLMCh3XjDhdV/xwW1jYe2ziBE/YpxLyh9fgy2rU7wRtOtQFzAhseqXGi1aRqqyGjjnLZ66X3nrB7NePOZVx3jwvdyJiB2wCWeTnP6MPLhYrfrk372e3/K4i49N87576go/3nzrnfAvBLAP5g/8NFinQwaD/K9z+BmGdLcs59DsBHAfym936n8sJIkQ4w3Szg2hbeLMA1ZW/j6bXFIpN2OOycuwkh7XzN3rYgDlVC4VKnHLToKwqnh/mwHT9AALXqdlsbJEp02nJMS8VUSwQ/f9ls7DsahHi0YW0njlmm4MKqHG/smId9cSHUCTRxp9GWvhsTm+Oy+kh6MKV5k16Dy0MRWbZ6JiItqdGgMVcAUfpMSUHeIdzgAPVfT+33Xe0zn1gMebYgwYMFOZacFpvhcs1cgCfT8ssdcn6FcgRzri3rEDdKUXGOFKmCbkvVVtvtq3beAPvCW/jeMG9U7P5VCOhVGN6uwk7Mm0gl86g01+7N32YPenDwdwma9bUBKcPLssN3lszzO23K8eeeNhvGHR3p9ME12/2fepSq0bYkISghyJ2FJVGm2ewd4GI4pD9R+KCjDduN7+iY4n+8L7/XvCnnL2uW0AJVrQ3cowbbtdeoSD0UiT2psK83QAVmZsqZUhvPOT12VJSG3POJjn3vHdb19kCxawdkUtexM31I0xvgHJFjRIpUQfHDiBSpgm5TzrdQkE4Y8r/Z0Lzk64SOhOC/NzwtKNVsDw/jzWX6zZ8vnV47YLBK8W8qfH+dxJS+lgmYEoLeBZKrlhWC//Q9Jrok2sHxVfNtnFgTpfn+0xaKXl+yfOrFRVG+L501jNxQZbagFbTU98GiVKOtBV8IxY+c82Ue9RplDB7JREx5acvEr5Hi4c4puyzuJIoMSAr0SPFyyT2DpgZcMuasL7MHbS3e0+8zCWk/0TJ/yKsLcu6VLotSMrmVhWxu/P1Q5BiRIlVQ/DAiRaqg2wyGEMQhFndUBKqoyzdnGNoLNDjvx7iGwWGu72RvR2y1SkP24Jw9fK8Ad3hVS+VOzJcw0ST9BoWEPLBgItDRNblmdc1KIN97/zsAAMdP3l+21TIRuR58wCL61wi9cLYjUDu7f2rljo+elL6bbQYsUMsR2ftTzQTk+zUlxD+nte7aJF49elpKIN952MSir78s6740oGQOep1yreU3o5vfG8ncWhRs6BX4wFOGYqsVrFK0b8/FDEnfDEo5UfG1yrcV8kgcoigVKdIPRbdV+U4q7MgBpWY22/ubn9+2te0NQNYCI6h0m9McKkoRJHScub2/h72KOcuxo7KLTndtaw27TEr+hVP33l0er6wI8BkXRLn7gUcAACfue6hs296QSq/1RcvGW7rjvvJ4uv6yzIcsFmPITj+l9Me6+h343k6VqxVUy3DUtx28ruHoy3e/u2wLXuf20EDh6uk5AMDT3zOM2ytU695DDQy0hddCJt3Y7k+q81jlPPAy6I9D0a3vIlGuR5UBkppyA0fecr0mOOxvALo2coxIkaoofhiRIlXQ7VW+r/FbdW2AvYkU88r3XlZbqZzPYdxqKAcZ91NydJTZAaz36d8aBdEtL4vfoX3I/A99VYoTyltYWrTwD6cZZDuvmf/hykvPSz/HjpVtq4dFOe9vvWpzIKzZtCF9bpmejXRDgpxX7zL/w0Rt/35GSqdTRZyqQ3EA5+FT75S/j//1sm26JaJd9/uG/bq2JKLSo8dsrV88axPaqaiK1Va5udUxETGMnJKiXZRzoxwM9mM4FaUcAU1oKmgVDFNd81uqQoPeiCLHiBSpgm4P4Foxn0nHVrgArsWI5IFuCEehVM4rrp/LvJPjRo3yjidUkD5wt4qdp0ZKdUvr2oVQcgCoaZ5zThivnKscavgNNk2JPfvnXwYAnGo8XrYdu/MeAEBGXK3VoeqwL4mZdkqArktHTsp4q8QxtPhLUrcdekGLsrAB4NAicauHPypjdywPvH9WwMxGWwbcFtZybM044jt79hC//Jrc08mcrVxD1QnsLdzmPGcTrmjxLmEvdouOA+CxrdW53lx/cp5QveQm2DftO+fbOfdR59zh/XcdKdLBpet+GM65VQD/BsD7AHzeOXfEOfcZ59yXnHO/fstnGCnSj4D2I0q9B8Df8d4/ox/Jh6GAa865zzrnznjvX3jDq/1erzSztMQFMavY8/v1MG4rf68ASOA2pxGMzRaBAQxMlApZeIVjxT4ocSYCZHrsEiqfq9lsBWXHBfEKADKde0YKeyi12904V7a1WtLn0prVqmPR58qrLwIAjhy2gjB3PiT+kHxg9f+ai/J7QaLiUO/ziZ/8lbJt+a53lcdJS0So2cD8E4mKnUndgh8LhbBJ2SdzyO7pKxpw+OI2F66XYxaRylIFJEpNtSwBG0WSlPwTpQhV9S4wsIVQKGF9I6L5fnK+/z8AcM79DIRrrMHSWiPgWqS3Je1L+XayTf8ygC3IRnxTgGtlPnWVR3vOMiu/U+2WEn92ToEq9vaT0kUBX5YvqilcC5t6swqUtirTckJWg0Q5D/czU0Ru74yzjMemiDs1G9bq1I9m3A0HBlfT3xZM1sPHH7C19C6Xx5deEe6yduKk/a4e7ysblk9d14y6FuECDxWk7FRChekJpqZQCBw/txvLugqOBtA1ciRCg/BwT6/JOGd3baef5LL+YmL3JJhZGQotFK4Hx1R5Mzm7bKptZpwoir1iRpmXfxMcY1/Ktxf6JIBvA/gpRMC1SG9z2o/y/Xedc39T/7sC4H+CiE+AAK6dvTVTixTpR0f7EaU+DeC3nXP/NYBnAfwegKedcycA/AKAJ/Y7WJVSHfwYHELsK0SuEC/HRcznXB/aTMg1JXtlv0lNbej5hEAKSBwI4sJczTfte5lAA0JF0Rl5lQcllqrNcTSi3/siLi00TYxZHojN/nBu5y0dEs83+02K7ZfL462e1uBbNuV7R+vVpeTvOPXuJwEArZbNO/gf+gODxykuvFQeJ2pAaNEcG2un5Le2Weu9Fpx3NbtRjpABjyva4qFLpvi/uiv3x9FafQgxoHqC4e4V9BA8eeoTp4GQ5NuYBjEWoPPkb0BQvBE/xn6U7y0IuFpJzrmnEAHXIr2N6UcPuBYp0luQbnlIiHMGGDDLVUwhGWg03ntNFTBComEdeSXasllRHIlFQRKZF8mkgwlZRuYq5Vb0vaRixZm7za8QRKkhJfuHMJKM0Pfg7BZnUzl3Qjb7/kjEgpXCMv1abRGH0oEVj88HFni4cUV8DMskI3od5hBlAq4evwcAsLBiItBQqyv1ulaNaOMFywRsdmQed977aNm29uCHAACNtdNl2+azfwgA6F4wPKzZzIIemw1Z42N3WchI93siNm6OCVfe/987AAAfLElEQVQqvAwJIQ0GzCoK2pwro6hWspyAnidTOS44/0PvTxqDCCNFenPo9oSdqwczfOBTioIulItQym/5ZVdm6L0BmlzpOJ8LGJzP+QVI8SfIekK7QZHqNdT3iaOi5B49fpQGVAMBsbfARVhpnvOWqz192DNfQ9jp0swU5EZTgv5mo0HZtrttAXxdrSXXWjbFdleV71rHAhR7XTmezow79rVt+8r5sq27Zd7yFQ1GbL7HFPt6R4rJ1Bcs2LCugYeTL/wvZdtwy3wthfoaVslg8fjdkpf+py8bt9oZC2dhxMK8dG5xPT3mGWr4oEquIeebMy8DQy3Ux+OvBQzwOoocI1KkCoofRqRIFXTrlW8Ye7P8iL1ogXMssAIqJxD7MTg4sCqGI4RtNEgZXmqLmLLeM9ibMTlRsizM1cZZVaicesNyOMaqNM/mEklkvOmUUP5ovjNtr1Ox94aKbjlB2PRfE4V2dMTEkEl3uzwOIRqOauIF/wtXHrp64fvSN4kcU0ULHFA1pvHIxl7VWn+e9sxwfUq1/OpqIOismXi1PmUQZp13butfUX/KA0fN0PCti7s6L87Wk2N+BnOPV8N+pmT4CH4MfmlCdFB4DyIYQqRIPyTdco7hQeHfFYqzp+Bw7DnaS/PVVjnQze1pa9V1h6fQ8PWrosSOyUvLyD31UplmJU7NfgRCVtdAwA0K2svVi1vLbL8ZECBZovvQ4UPmnT66IhxsTN7nc1+W3OqlhwyYrbZqiv9QK8Xu7FANvrasZ4nwWTcvX5C+KZCx3xeFPqMQ8hHtvIc0mHE0sHW1l0T5Zi6SNARX98hjHy/bJgPjwhef/Q8AAEfjXN6Q+T5wv+H3XxkJdzx71ThiWfKBngvZSjDTNU5mNp/wOPn9CNw/OOff9CDCSJF+3Ch+GJEiVdBtRSI0rbpKpNp77Co0ahaVSBfEYa03d88dbH8XMeXyptnNkYtYsbhgHtlWk8r0tmSvSMhjvaJQMTXO9ZjsVWJ3tXrSAhWXZ/a9pJA7x0+eKtt+8gkJVJ6d+0rZduV5EUNYke7tmGgzU0fQYGh+jFyDFZPktbKtrWUAtrfs2lRheGpNm2NOt7m/I/fqykUT7eptEZsaLRMBa5q5ly6YWHTHT/0X5fFY/RPrzz9Ttt3/HskvqR271/pZVLTEZ75Ztl3cDPeUcyvsGQUxeGfIHnQ9z5aCmorAyf7dFyVFjhEpUgXdJnOtUBV2baC5DL6q/N3gDactYZnwTu8/ISbVn37CMGAvbkjszl1HbXdcaCtKeY1CsSkSx2u+cUvPA4AlDe+ejizLrtDQaTYonL8sMUxLHfNYtwhp/Ogx2V0feuT9ZduJex+T+Ry18Y50pO9xzzjd2VfN8z1WjpGRy36mJudel82wMo8hhb6nWhimDtuB88L6CbFUQ1K+R11py2p2HzMNE885iqlBxW9+8pcAAO0Tlk8+03u2/Zpxh2WNqTq0YKEPl69qCDlVGMqIWwfz63aF2T+hF6QeTO+BndwA54gcI1KkCoofRqRIFXTr/RjOEuar7MiBG1bGd1VcEHBIAeB+CgN//+Oi0C62LW/KlYAFVssulE1muHxfmKgxUc/wjMSLq1dEpJlRkZhGQ24d15jb0cy6na6dd8cx8z88+j4J3374MROllpZlbm7VFFuMZbz+C/++bKo3TfRbOywi2bs+8PNl212nRbFlGJphT+5FTrmO07HM7fJrlhF47tzZ8rirAY4F1dOrNWTslLLsDPvX+p6NbN1ez62RIeLlz/8eAOCVZy3MfazjvHzRnltXb2lK/qCMnodhIe9Ns0w57UCnWCQ37siIHCNSpAraL3zOMQB/6L1/zDn3GQAPA/gD7/1vXPda0IcaYlfo9zJavCrWiT7xFVXOnnivFWJ58i88Uh7PxrLjvHb+UtkW8pqXF23XDkkutZTjjCjhSZNjNq8a4Fh3oEkwlN/d0tJfA/L23nU4FG+xNfzcx36uPP6Zn5Udvt2miieFmFwTMga0Tv4EAGCy/v2y7dT9hiX7l+75CADg2Am7F06V/Cnt9FldEos4oWdnU+7P5pULZduQYqV2dsXAsH3Fko6mE+GECSUOBQWYS4BNh6b4BwX74tc+V7Z95ytikt7apbB7zbA6v2197+h0HKOZ00ujBXMxmZlhIwDlsXEmmGtzjT+/EavtfjnGPwTQcs59HIpCCOBe59yZ61wXKdKBpP3A53wYQB/AJQBPYS8KYdU1n3DOfdU599WiqDojUqS3Nl1TlHLO1QH8fQD/CQQ2p4ProBAC80iEWeq886/TeuYQ9vcGjKXKFk/fZeLDz39AvKWHVw3SfnPd8qC3tGhLf2hsekmhZBwp17nWoPM5Kd80dpKImLPUtj0jYKj2B+zv0Fxtmw6WO8La76L6dX/1Fw0jdnVFvO352EQOVxcR0VH+drogfpPOvR8s27pnzfY/eElywc9fJt9GIvOpHTJ/SBkuTlmEO9tyf174vuVqb+2aONhakLlxxuC4vKcM6b83yy6hqqiDDUFtXV/fKNs2+nLuTtdEpEUNqPylv/6hsu3CumQCvnTOct7PnzfRdmtLxGaG1wlHnFEZpL1Qg/AGEviuyzF+DcCnvPch9LGHiEIY6ceArqd8fwTAh51znwTwKIBTAF4F8AwEhfB717g2UqQDS9f8MLz3PxOOnXNfAPBXAHzxhlAIHUpTQZnpwKEhJRizMZ93PyR2+o990ESSVEM1Xr1g7HVAYpNTAOAlgvdPFAwnpywyOBGVcgpK2yVA5XZLfm9Stl6wQB1Z4ao+IlZMpiaShUpBT330L5Zth9bMhxKCIl2y1/6eEzpfeacWDbT5hR/86/L4la+LWHXq4cfKtvULP5B+NtjaJvMZE/rE+lURlV44Z1apS1dNtOvPZOxHDD0HrUURaTNOSPGy7iK3XI/Z2HIqhj057g8pkSITsTGnAM3j94rl7MwDd5Vt952Rdf/UT9q8RyN7Z777wlkAwO/8ngUohrwetkoFgOpwrS/2L0vtWxTy3j/lvd+FKODPAPhQRCGM9HalG/Z83zAKocceracqC++h+82L/cH3PQwAGPfNvr6+LnZ1R57WVoMqopbKK0HV6w6RF6bsDUayO/IOPSF4mZZXZZiC7JxC3nMZgFQBwg5RsZR7HpQ6ese1hp70Ywpr8AMkBMFfzoNS1EYayt7fNjiaWUFZeFps5vKG+RrOb8kO3b3Yo2s02K5r175ySZTY9W0LdOwNSRlekRzuOx94smxbXjuu86fAQ/UbjXrGJbbXz5bHO7syj+7Axn51XTh8c8m8/He/U4wqSWKcN1P8XFe359ts2Nh3Hpd3ZUJ54uE94kDV7kjhijTYcnYrOEakSD9OFD+MSJEq6NZn8DlTtg2wwH5e1lCP99xn9veeQsV0u4aqF4LVWk2L+eesrtlMFGg/M/HL6Xc/V5pYlfQxlRye0PEsF/bdqJuItLQkcC+Hjx6jNgnhWD5kVYTS+n0AgFHPbO75zAANQg5HPrU5TjTwrsjNADDYER/A2T//D2Xb9lXzK0wUtnE4svCXe86IEuuo5l9ZNrhmWY1//o2vAgBeesmCCJHZWn/5v/zvAADvfb/5FRoaRJhT4GW/K2vsbljG4NV1E+02Lom766XzJg52M7m3H/iw2WxOnFI8XKpCNdMQlNnYjCvDXctNeeUHUoNwe9fuYzDe1ClvY6Et92lDY0zyN9GPESnSjyXdeo7hjVME22xK5sr7T8qOW8ttR+h1ZYceUk7zQkd2mzmk9AmFmHstSsJu9aAs007R0gr3zaZxqAGFSx9XrnD3KUMNP3nqHgDA2uEjZVua6VqalvNca0iwImPX1hrGUaZaPXWwYx77XE3Oy2TWbWjRlfZjNoeHH32f9eMVsKwgU7HulI6DI2eqYGem7D54RhTXr/ypcaPO6p3l8em7ZQ2Xz/552XbkhMxjOjauNlal+/zzf1q2bREG7oUNeZ6rd91Xtv3sX5M1nDhp6OvTXNbf7xpn2bkqz3XrsrWtX7K+v/5tae+PKVqzBFez539kVfPbNQJxQEGO16PIMSJFqqD4YUSKVEG3BT7Hvy51arFjdvwTh0Tp2t41sSjxwrKTpmGczhR1Ls9NvCrI/5BpoN9kar+H4vNN8nc0NGgv2OYBYHHRfj98SBTVtbVlukZEFtbht66cBQD0KSCw1RQWf+JuEx8Wl01EGu6KQjodfIOuEXY/GTPEj4gpreJc2eQTU2w7NRHfZg0Sr/KQM0LRAKn0PStMcT92XNb/0x96vGz71tdeLI+f/oN/CQA4ftzuz5l3yzhFbiJnoaWEX/nB82Xb5o75NA6fkoyEdz72jrJteUnuxWRkz3prQ+7Ji99/rmx77bysgf0vI6owdPGK3PMaiU0zzQthUSoEUS605DVP51/Da1LkGJEiVdBtMdeGLzUgch9apJgbzVEewcywQRle9rb7SwQ8kHjbtRLaAobKSVghW12Q5XU69v23FO18acl28uUFMxWGGClHeLcT3a22J5Qn/bLs5tv9dbpWY4FyMyM2m6Z85+NwLuG0Qkyh4wFxjFSuabRsV09yK/Ti1QteTKwflwqn8/RIPfS8wnbo1ItpNssI4/aKxZ+98B2JuepeNAPB+Ze/AwCoLZhZ99ARUaBnNdvJ3/nET5THp05LHFc+Ie99/+sAgPULxsF+8KKYnC9ctLXs7AZsXut7SOXJNrX662LT1tofyTWMCRCYx+wmkoIix4gUqYLihxEpUgXdFiTCLCSqq0OhRQXucp1C4ikbKxHxYmdorDT3Cl/vrK1O1e77k1zPM6dFSNhn5d+p95ULvgwoiM45YbusaDbba3qejf3t50TkyGrmD0k0FDt41wHgyBHzfayutHWO5p/o7j6vY5iy22oLel9OiuSU4WNqYpRIawbykAZRc67eoMwtK+w8r/2Mr1gqzfaO+ScubMhxQf6Qd50WsfOR995Tth3TQL6kRsXsYaH6LunpX0txHA1lPdsbJjZtXhbxsrfLcEZy3piAHc6vm1jdm2qRHAoKDM+dshfKYjKF+tFuBRhCpEg/VhQ/jEiRKug2lQGYL/3LloPJNGReUel5ZZHDIdXGS0SMqaMaxMBprEidagPkijs0oLyONJNjTqSfElreRBPnR1SFqNaT49descT+zc2Bnm8sfqR19MrgPQCPDiwQcqIg1MHCBgCzsYofLRO5pmNZy3hs4k4rf7Y8LlIVxWqWU1FryPUcjJhqaEqamAUuBGZ+80tftLZdm+PpByQ85Imffqpse9e7xW9Sr5sI1Kgv6FrsIUzIhzIZi7WpGNu96I5kHttDA7mYpbKGNDPrXjGWNj/mnBF7XodW5XpOr9jYkOsbFI5TgiDo/28EDOF6KCEZgJf0HwD8bQC/COAvAviK9/6T+x8qUqSDQ9fjGO8B8L977/8uADjnfgKCJfU+AP/AOfcR7/0fX38Yza7Sjz4UggeA6USVN1K+QzZWUMIAYKRFWVyNoe8pg0t3gwaFog+Hco33tmu5nuzqk5FxqNGO2dUPHzuuczC/yrbCy1zdth2z1dibFVYk0ufROxie3jzjg75wh4zgbJAJ5u5kalxkOr2k8zeOkSSMCyu/J5RvXWvJTp+m5jcpct15KRR/sCs+i9XDdt5HPmb+ktVjwj0Xl42LFjPxRUwKKgNQk2DLjJTr8dTmO+qdlTVs232+fE762d4yv8pIC/Dk5DdyIeKhbu/EGuXbBw1gq2frt8oRpJDrszEbxpuXwfcEgL/knPuKQnP+LID/03vvAfy/AD5QddEc4NpNVLOJFOlHTdf7MP4MwEe89+8DUINgSjHg2rGqi7z3n/beP+69fzy5gfiUSJHeKnQ9Uerb3vvAr74K+ziAGwBcC/6LNIDrEhuZKLRLTmJTEJEyMkqPNf2qwUXfx3uT4VkZnIXMPOJaMxUr0tzEp0ZhCqufiOjTOXK6bBupEshBhK2miFLtjoExv+cO8RccOUTBbxTqkSh0TzYzsSlNFYx5aop9MRVRI6kZpEzXW1BjPZWxW417aK1aNJ52IqePKicgBZfKeWfuN1GxmZHSnEhoCYNFFIrkWBCi41RLIvgai1wmno2nMt8Ll0y8Wr8iWX99UvaLQoEmSAScqHh1adPWUqNxvI59ddP6CSAIzTplderrMWWU7X3S9V7sf+ace8RJDulfhUB0BrzaRwCcveERI0U6AHQ9jvE/APiXEO35cwB+AwK49o8A/Lz+uy6FDTvA2cyo4nwWwqXpGw0BYXOZd+rlnjU4ANE4RvDophxb7EJxF2sKhV7SiSmkBeVJF9vS3hv+oGxLNefZZbzTS9+HDpnp8fhh4Rh14lrTkQ0+ToR7JKllsBUK7TPcpaw+hcdpLRrHyChvG6nsxn5k8ym6Z7U/CsWviXk0yWgn1+lMRsbpxhmVIFgWTpk682InaQBXMwPAVO/jmEDWJhTot3FRdvNnn/+WXTNQZZhi+nwqnCuhrM4Ai7O+Tc+IOJjX1ILhxNpWlmW+Cy17PwYTN9ffjdD1kAifhVimSnLOfQTAfwTgH3nvX668MFKkA043A7g2BPC7t2AukSK9Zei2eL6D0pqpV5phTLyyuQkh9mUBVY4USa9uy8nEzquRIp5mMz3PWPJsJuPNSLFPnIyX5OQhT82vMNai6o2+KeftBQk8rK1YoF9dlcWCIGUG6qmfUlZf4QnkYUe8s8trJgLUW0s6rhkAAojDmLzvE/ICh+Mss6w+P93cc02iYkp79cGyze4JgTSQrBliNLMaoy7K+tPMfBYhpnFKGZP9rolVu1s9/d3EvbE+69rcc9Ux6FnWNauRDQDdPmVuanDgjJ714TWZ2+HDNt5gGERpOd+5/ZtIY6xUpEgVdBs4hiuBzwLH6I1I+5rIljGmWm4BbodxSFsaXzUhy1tSUUU0pxioYCam2iYIwOe+ILPekMyCikm7QjFXiSpxjgDJp+rl3qYsszwXk2ujToVqyIO+ovnf/b7t0DPNPExSi2fyei96m6aQu8TmU2uJIl7PjMs0Nd7LUR58vydcr0gtNL7W0HHIYz0ZGHAbdoWrpTUCtlOzeeroRjrNPCRImquXLctw66pwynpi/aCuYeAcsaDvBleEXVkWDrW8ZLn4G1sGYhe4TL1h3OHk8UP614wG65sy3tVtucdcnOd6FDlGpEgVFD+MSJEq6DaFnYsoUigP3CZ7d+Cq7PkOUlVKCnlQnOq5tdXJgz4Kx97ElHY9eNrJ96H99KcMF2+K3VJbFPEWlQFItTRAOqVi71rDj1EVRyPxIGeZzfHKtlm07z4lAYPtlim+zaYo0vWa9V1MRQxJyW/COL3TmXjGx7SskSrvBcELdXdF/BiOTSRrtmXeORWTGfasiIwvZL7tRTM0QMsWJLC+s6ZEA40opH/j/As29o7cC0rCg9eHPVfzMA1ithlAlpbFIHH/aQvF/97LJkqFdyFADwHAoVUREevk53LY2TPefilyjEiRKuj2cAxlBqOQgEP22uAEd+TmDjrSjKq9Bi7jyW2aJezRVHMubVGNWij48vqzgIwUe0+Kf0hUGtZ4Ow6h0XvRtdnTnqmH2FNxm+6IdzqZz1LHOEZQIOuZzbtek525s2je7npKCOp95Q6gUGz1ShfE6aYa9zVZ/37Ztrgqc0xS8nyPiFtpstFgSNxaK9n6GZmhFc5nOLb7uEnI5xOFMZqlFuMVIh+4iG8orMPV1zoaf3b/fQR69ydkXtepddrGZRpqXp5OrfPBUNY6GITktP3D6ESOESlSBcUPI1KkCrq9yreKUFNSfMsMuLm6fPI3IU+lFhPFdGBsfzgz1rik4catjELVdZw6Jcw5H8KTbQ7NqbHpkXqBR4ldNNE6cqm5CFBXMa3ZJNFOxStHED+Tic33yob4FYZdm2O9qV51mmRNRbb6FolXqYlxwY/hKb87SAlzVUvVpzGZ2hzHhYhkdSpPMByZV92p0ydNTGyaKOIh52AX6sUejW1eIecdAKYq0iYNCoOvie9kTAaCyUCLxHibz5KKdseO2bWn7rTAy5dfVWzfOhsn5O+Aav51e1p2QZ9pcQNaeOQYkSJVUPwwIkWqoNskSgmFIuWcwefzeYgTwOzUi4tmdZmpKWI6M/t7b2JXDbW9RbkVAw2YW2B2rqLCkWUTn9p1Fr/U+kWmkyIELqYsxsk1HOsfxMJ2g+ZFINNe8yx6Zn5HTZH86g1rzBIRAbLMRBcCXUSSyu8ceBfyGVLy46QqVs4KE9OGU7E6FWAQZbN4jaYyZpMy5jotEfcc3ZOZ5mPMaH25I/FMq0fVpgZ8UO+EkCDLGFxQWMoWZd5laqFbXT5Ztn30Q1bu+t/8oVRxqlOgY8jS2+2aCNhTH0sIPPWxnHGkSD8c3dbCMb70gPNvrz8wn4Yju/Pq0oJeayf2BxSKreHfPfKqD3UX2abc8DUtItI2/a/0XcixYruS0hz2cs8FANXZ0h9a2/ZA/Q8NVgoZzkW5zJR3epljjTzfjZq2EfdjX4xTX05GmnaifpeUxqs1ZDf2dM/STHbwbte4xA4prNMi5OfbDVpZUIWcQPGcKteOsgN9YusOlXJ3B8bBFryMmZHSvLQo3u3lZfNyL3XE892mgj8PnDafxhOPSWbjpcvG9Xo94RTbu2aQCKHqwQhzI4g1kWNEilRB+/4wnHOfcs79ZT3+jHPuS865X791U4sU6UdH+xKlnHMfAHDce//7zrmPA0i990865z7rnDvjvX/hjS9G6aMw3W2vLOUqwjZ6FKAW/AutloUyLC4s7rlmQOLVYCTXj2m8KwMRAQYUENgm30dAP0w4fCCUE+A5hoA4AmQI6uNobAYCXtdExyyatoYAd8M+krGKVfUai1KE1Afpv0Y5IwH7NyHIoSygB82JqWIAyLlWIQEf5LrIHbr3yEVMaZKy69QY4uqcIGOHmd6zmTexqdeXfpYblKOh4SaNFsH5tEWsGg9Ncb+yYbke33xWqlkdWrF5X92W597tk3FGQ3mGpUEF+6brcgwnAT7/K4Czzrn/GMBTAH5bf/4jGJwOX2NIhBGKMNIBpP1wjL8J4DkAvwkBdf4kgM/ob5sA3vv6C7z3nwbwaQCo1ZKybkv4Ror5KLJwFV0vf3NqC4Bi0xlxhIHtep22KGqrBGfTnoRabqZoBnPvNgUy9sgM29CtosMQP6kcE3MoC7mQfoxF9V4PKc0wo+i4EPyWEwBYME1zOGSwSE9oXlmNFGjN5ptSqL4LBg4yEEyLoMTbLppo+HpBUQNJSji+Rcg3p3ufKQI834AQrMlx5QkV7Sk5F4fLa3QC5e2PRjK3Gd2TEGw6ndq929m2HPyTx5Z1jsbVLl3VgEky4feVYwTjSnED2LX7+TAeA/Bp7/0l59w/B/BTuAk0wkiRDhLt56V+EcC9evw4gHsQ0Qgjvc1pPxzjMwA+65z7FQh27VMAPuecOwHgFyCI6PuiPIgnc7p3BXtTCYHhToIYxjX2WPHd6Yn9OijcANDQDK+EIOaDyNXrcwAe1ePTgS6Q7r2s8tJCZmPXQy4IY+6qNFCQmJYkLAKFeZM3XBdL7odQ4Rk5gQ9MyMMcpDPuO8lCACN5g1V02aVyAQ11obNxQaDCwnLk+sRzcR85d8IRCyqmOirUktTYf6NGDEprmSoSRY3AIGoNEYevbplPIq2LQNJsmXJdEKLF8WMSRLm1bcr51W0xKgwJCmigoBt5sfe9ux5d98Pw3ncB/DVuc849BeCjAH7Te79TdV2kSAeZbsrz7b3fglmmrkOuhF8pfABFo2y90OfcFXvbqsKqOesr9DkmPNNZ8M6yAqwacLNpXtVQqRUABgNV4mj0DfWcbndt511SLX2BYnxC3xPiQLMRe8t1F6V1ZZoBWE/2KtJzbIQ5k2YcMrJ5Mgto73ZJAJcD5cn3tNhKg5/8nEdfM+pop5+oIj5nYQypA7nt0ElO5twkcAzi+rquISnfoRLu9pbFTzU0JN4RpFC9QSZuH0zOxEVLTzvFbul8Hec07JOi4hwpUgXFDyNSpApy/mawRW5kAOc2AJwDcBjAleucflAoruWtR/tZx93e+yPXOQfAbfgwyoGc+6r3/vHrn/nWp7iWtx692euIolSkSBUUP4xIkSrodn4Yn76NY91qimt569Gbuo7bpmNEinSQKIpSkSJVUPwwIkWqoFv+YRz0NFjn3LJz7v9xzv2Rc+7/cs7V3wZrOuac+4YeH/S13JKU61v6YXAaLIB7nXNnbuV4t4j+BoDf8t5/DMAlAL+Cg7+mfwigddCfzxulXONNWMut5hhP4TppsG918t5/ynv/b/W/RwD8Kg7wmpxzHwbQh3zkT+GAruVmUq5vhG71h9EBELLYNwEcu8Xj3TJyzj0JYBXAqziga3LO1QH8fQC/pk0H+flwyvX7ICnXb9pabvWH0cPbIA3WObcG4B8D+Fs42Gv6NQCf8t6HrKCDvJYy5RrAPwfwNN7EtdzqG/E1HPA0WN1lfwfA3/Pen8PBXtNHAHzSOfcFAI8C+Ms4uGu5pSnXt9TB55xbAvBFAP8OmgZ70DL+nHP/LYD/EcC3tOl/A/B3cIDXBAD6cfwVHNDn45xbBPBZiMhUgxhFPoc3aS23I+x8FZIG+7SyvQNPb6c1xbW8QV8xJCRSpL10kJStSJFuG8UPI1KkCoofRqRIFRQ/jEiRKih+GJEiVdD/D8vAMGphx4TmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#展示图片\n",
    "index = 7\n",
    "plt.figure(figsize=(3,3))\n",
    "plt.imshow(train_x_orig[index])\n",
    "print (\"y = \" + str(train_y[0,index]) + \". It's a \" + classes[train_y[0,index]].decode(\"utf-8\") +  \" picture.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((209, 64, 64, 3), (1, 209), (50, 64, 64, 3), (1, 50), (2,))"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_x_orig.shape, train_y.shape, test_x_orig.shape, test_y.shape, classes.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据处理\n",
    "<img src=\"images/imvectorkiank.png\" style=\"width:450px;height:300px;\">\n",
    "\n",
    "### 数据规整"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T \n",
    "test_x_flatten = test_x_orig.reshape(test_x_orig.shape[0], -1).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((12288, 209), (12288, 50))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_x_flatten.shape,test_x_flatten.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据标准化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x = train_x_flatten/255.\n",
    "test_x = test_x_flatten/255."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1.0, 1.0)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_x.max(),test_x.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## $L$层神经网络模型\n",
    "### 训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def L_layer_model(X, Y, layers_dims, learning_rate = 0.0075, num_iterations = 2000, print_cost=False):\n",
    "    \"\"\"\n",
    "    训练𝐿 层神经网络模型    \n",
    "    Arguments:\n",
    "    X -- 特征数据集, 形状=(特征数,样本数)\n",
    "    Y -- 真实标签值（0 or 1）, 形状=(标签维数=1, 样本数=m)\n",
    "    layers_dims -- (list) 从0到L，每一层的特征(节点)数\n",
    "    learning_rate -- 梯度下降时设置的学习率\n",
    "    num_iterations -- 迭代次数\n",
    "    print_cost -- 是否展示成本值的变动情况，如果展示，每一百次采集一个点    \n",
    "    Returns:\n",
    "    parameters -- 模型学习到的最优参数，可以用作预测.\n",
    "    \"\"\"\n",
    "    np.random.seed(1)\n",
    "    costs = []# 保存每一次梯度下降时的成本值    \n",
    "    # 参数初始化.\n",
    "    parameters = initialize_parameters_deep(layers_dims)    \n",
    "    # 循环梯度下降\n",
    "    for i in range(0, num_iterations):\n",
    "        # 前向传播\n",
    "        AL, caches = L_model_forward(X, parameters)\n",
    "        # 成本值\n",
    "        cost = compute_cost(AL, Y)\n",
    "        # 后向传播\n",
    "        grads = L_model_backward(AL, Y, caches)\n",
    "        # 梯度下降-更新参数.\n",
    "        parameters = update_parameters(parameters, grads, learning_rate)\n",
    "        # Print the cost every 100 training example\n",
    "        if print_cost and i % 100 == 0:\n",
    "            print (\"Cost after iteration %i: %f\" %(i, cost))\n",
    "        if print_cost and i % 100 == 0:\n",
    "            costs.append(cost)            \n",
    "    # 展示成本值的下降\n",
    "    plt.figure(figsize=(5,4))\n",
    "    plt.plot(np.squeeze(costs))\n",
    "    plt.ylabel('成本值')\n",
    "    plt.xlabel('迭代(100次)')\n",
    "    plt.title(\"学习率=\" + str(learning_rate))\n",
    "    plt.show()    \n",
    "    return parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(X, y, parameters):\n",
    "    \"\"\"\n",
    "    使用𝐿 层神经网络模型训练的参数进行预测    \n",
    "    Arguments:\n",
    "    X -- 测试特征数据\n",
    "    y -- 测试标签数据\n",
    "    parameters -- 训练模型得到的最优参数    \n",
    "    Returns:\n",
    "    p -- 测试集预测准确率\n",
    "    \"\"\"    \n",
    "    m = X.shape[1]\n",
    "    n = len(parameters) // 2 # 深层神经网络模型的层数\n",
    "    p = np.zeros((1,m))\n",
    "    probas, caches = L_model_forward(X, parameters)\n",
    "    # 预测值离散化\n",
    "    for i in range(0, probas.shape[1]):\n",
    "        if probas[0,i] > 0.5:\n",
    "            p[0,i] = 1\n",
    "        else:\n",
    "            p[0,i] = 0\n",
    "    print(\"准确率:\"  + str(np.sum((p == y)/m)))\n",
    "    return p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 两层神经网络的应用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost after iteration 0: 0.695046\n",
      "Cost after iteration 100: 0.589260\n",
      "Cost after iteration 200: 0.523261\n",
      "Cost after iteration 300: 0.449769\n",
      "Cost after iteration 400: 0.420900\n",
      "Cost after iteration 500: 0.372464\n",
      "Cost after iteration 600: 0.347421\n",
      "Cost after iteration 700: 0.317192\n",
      "Cost after iteration 800: 0.266438\n",
      "Cost after iteration 900: 0.219914\n",
      "Cost after iteration 1000: 0.143579\n",
      "Cost after iteration 1100: 0.453092\n",
      "Cost after iteration 1200: 0.094994\n",
      "Cost after iteration 1300: 0.080141\n",
      "Cost after iteration 1400: 0.069402\n",
      "Cost after iteration 1500: 0.060217\n",
      "Cost after iteration 1600: 0.053274\n",
      "Cost after iteration 1700: 0.047629\n",
      "Cost after iteration 1800: 0.042976\n",
      "Cost after iteration 1900: 0.039036\n",
      "Cost after iteration 2000: 0.035683\n",
      "Cost after iteration 2100: 0.032915\n",
      "Cost after iteration 2200: 0.030472\n",
      "Cost after iteration 2300: 0.028388\n",
      "Cost after iteration 2400: 0.026615\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUwAAAETCAYAAABdtj99AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VOXZ//HPlcm+EsjCvi8uCIKooIjgQrXVFrEurVW7iVu1dnmqrfZ5fk+r3bRq1WJrtX2s1gWXqtW6YUVQQQ0KKOICyCIJEAhZyZ7r98c5A2GYJJNkzpxMcr1fr7yY5Z4z1xDmy33Oue/7iKpijDGmYwl+F2CMMfHCAtMYYyJkgWmMMRGywDTGmAhZYBpjTIQsME1UiMgI989cEbmq1eOniMi1UXyf/xGRb0Rre8Z0hgWm6TYRGQe8LSLHAfXAFSLyTffp64GkkPYfiMgmEflYRD4K+dktIjNbtb1PRJaKyBIRWQJcBNwcvC8iy0TknxHUeJmIbBeRIhEZ1UHbgIj8RURKReQxEUlxH88QkadEZKeILGzVdnvIT4v73NdEpLLV49dE+FdqeqhEvwsw8U9VPxWRS4ARqvqm2wOsEJHTAQFuCmk/UUTSVXVv6LZEpAiobPXQTcBeVd0uIuOBe4FTgQRVrRWRw4Hm9uoTkSOAnwNTgRHAXcCX2nnJZcBIYAjwI+D7wO+A/wfsAQYCC0XkXFVd5N4Pvtdc9/UAk4FLVfXh9uoz8cN6mKZbRCRZRJqBXwDXisgq4D7gceDXQD/gPRFpFpEk9zXpwFoR+YWIpIZsMgVoHaTHAy+IyGjgduBnwALg/9ye7SJgirvdp8P09n4FzAP+rqrFqrocyBORjHY+1nzgFlVtABYCX2n1+G9UtcV9fF6Y117v/l3g1rWqnfcxccZ6mCYaqnB6b0lAg4aZPiYi5cHbqrpXRGbghM4qEZmrqlvcp1OAulZtHxCRT4ATgMHA74FS4DzgEGCBqr7htg0GW+h7/xlY0uqhYpye5odtfJ6hwBp3mxUiku8+ngd84t7eDIwNeZ+pQLOqBkNyCvCk+/pXgG+F61Wb+GE9TNNdjcAk4ChgJVDkHidcKSL1IqLuCaHRqtoYfJGqblfV+Ti7ucWttpdMq8AUkSyccPwYp4e5CqeH+SqQCvzG3eVuT4ADd/NrcHq+kbYPdiz2tvrPINw2LgX+7NadAfyT/UGfBFzRQZ2mh7PANN2iji2q+o6qTlTVo4Dv4gTpz3GCcrOqlrWxiaeAC0Tkafd+Cs6JI0QkACwDPgLG4RxPPBL4P+Az4Czgv9z3a2+XfA8Hhlsa0NLOxwptHzxs0CIiEm4bIpIMnAE84/691Kjqpaq6y921vxc4qZ33NHHAdslNt7hnnJfh9LiCva90IBcndL7tZkw6cKKqfua+7nDgmzi9x/8Av3Vf23qXXIA5qrrHvf+AiNwIbMfpybWoarPbm01sZ5f8PJye3j/cwJsKbGvnYxUBM4DH3c8X3I1eg9OTLgKmAZ+3es3JwFuqWuu+5whgvKq+7D4/hPZD2sQBC0zTLW4ADm39mIjMBq5R1YNOiojIAOA1YADwB2CSqpa3ahJotev+JeA6Ealv9fwooAH4qrs9cHah/wg80kaZzwO3ish/gInAblXd2s7Hehj4s4iUAtfg9IIBHgJuF5HrcM7eP9TqNafjBH9rd4tIsFf5PZyz8yaOWWCaqHIDcTj7e5sHUNXd7sD2N1U1uOudgnOMbzLu7rjb9mng6davD/YwVTXi8FHVShH5KnCzu/0L3G0dB1yvql8Kab9URG7BCfQ1wP+6jz8gIkOBu3HC8U+tXnYi8PdW29gsIjcD7+H0UP+KM3rAxDGx9TBNNLmDyMfhDL95MMLXHImzm/s5cLuq3t5O29/iBOZt0ajXmM6wwDQ9gogkuOMbjemxLDCNMSZCNqzIGGMiZIFpjDERssA0xpgIxdWwory8PB05cqTfZRhjepmVK1fuUtX8jtrFVWCOHDmSoqIiv8swxvQyIrI5kna2S26MMRGywDTGmAhZYBpjTIQsMI0xJkKeBaZ78arlInJDG89f3upCVqvcVbGNMabH8iQwRWQ+zjJdM4DR7rVXDqCqd6vqbFWdjbOe4l+8qMUYY6LFqx7mbJyLUwG8BMxsq6GIDAEKVTXseCERWeBe8qCotLQ04gJUlU93VGFz5Y0x0eJVYGawf0XrMqCwnbZX4qwvGJaq3qOq01R1Wn5+h+NK93nu/RJOvW0p72+riPg1xhjTHq8CsxrnmicAmW29j4gkAHM48Ip+UXHC2HySAsKza0qivWljTB/lVWCuZP9u+GRgUxvtTsC5DkrU95tz0pOYOTaP59aU2G65MSYqvArMp4ALReRW4FxgrXtpgVBfAJZ6VANnTBrMtvJa3tta3nFjY4zpgCeBqaqVOCd+VuBc9W+1qh40vEhVf6aqT3pRA8CphxeSHEjgOdstN8ZEgWfjMFV1j6ouUtXtXr1HR7JTk5g1Pp/n1pTQ0mK75caY7un1M33OmDSI7ZV1vLtlT8eNjTGmHb0+ME85rJDkxAQ7W26M6bZeH5iZKYnMmZDPv98vodl2y40x3dDrAxOcs+U7q+p5Z1OZ36UYY+JYnwjMkw4pIDXJzpYbY7qnTwRmRkoiJx9SyPMflNDU3OJ3OcaYONUnAhPgS5MGsau6gbc/s91yY0zX9JnAnDOhgPTkAP+y3XJjTBf1mcBMSw5w8qGFvGC75caYLuozgQnOIPY9ext5c8Nuv0sxxsShPhWYJ47PJyM5YGfLjTFd0qcCMzUpwKmHFfLC2u00NNluuTGmc/pUYIIziL2itpE3NuzyuxRjTJzpc4F5wvg8slITeXa17ZYbYzqnzwVmSmKAuYcN5KUPt1Pf1Ox3OcaYONLnAhOcs+VVdU0s+8R2y40xkeuTgXn82Dxy0pJ47n3bLTfGRK5PBmZyYgJfOLyQlz/cQV2j7ZYbYyLTJwMTnLPl1fVNvPZJqd+lGGPiRJ8NzBljBpCbnmQrsRtjIuZZYIrIfSKyXEQOulpkSLuFInKmV3W0JSmQwGkTB/HKuh3UNthuuTGmY54EpojMBwKqOgMYLSLj2mh3AjBQVf/lRR0dOWPSIPY2NPPqxzv9eHtjTJzxqoc5G1jk3n4JmBnaQESSgL8Am0TkKx7V0a5jR/UnLzPZ5pYbYyLiVWBmANvc22VAYZg2FwEfAr8DjhGRq8JtSEQWiEiRiBSVlkb3BE1iIIHTJg7klY92UFPfFNVtG2N6H68CsxpIc29ntvE+U4B7VHU78CAwJ9yGVPUeVZ2mqtPy8/OjXugZkwZT19jCfz6y3XJjTPu8CsyV7N8NnwxsCtNmPTDavT0N2OxRLe06emR/8rNSeHrVto4bG2P6NK8C8yngQhG5FTgXWCsiN4a0uQ+YIyJLgSuAWzyqpV2BBOFrRw9j8bqdrPm83I8SjDFxwpPAVNVKnBM/K4A5qrpaVW8IaVOlqueo6ixVnaGqvnXxLpk1mv4Zyfzm+Y9QVb/KMMb0cJ6Nw1TVPaq6yD1G2aNlpSZx1UljeXPDbpZ+agtyGGPC67MzfUJdcOwIhvVP4zfPf0RLi/UyjTEHs8B0JScm8OO5E1hXUsnTq+0EkDHmYBaYrZw5aTATh2Rzy4uf2CpGxpiDWGC2kpAgXHfaoWwrr+XBFb6McjLG9GAWmCFmjsvjhHF53PXqeipqG/0uxxjTg1hghnHd6YdQvreRP722we9SjDE9iAVmGIcPzmHekYP56+ufsb2izu9yjDE9hAVmG340dwKqcNvLn/hdijGmh7DAbMOw/ulcOGMEj63cyqc7qvwuxxjTA1hgtuPKOWPJSE7kty987HcpxpgewAKzHf0zkrls9hgWr9vBO5vK/C7HGOMzC8wOfPv4URRmp/Drf6+zhTmM6eMsMDuQlhzgB6eM590t5by4doff5RhjfGSBGYGvHjWUsQWZ/O7Fj2hqbvG7HGOMTywwI5AYSOAnX5jAxtIaFhV97nc5xhifWGBG6NTDCpk2IpfbFn/C3ga7YJoxfZEFZoREhJ9+8RBKq+r5+3JbmMOYvsgCsxOOGtGfE8blce+yjdQ22PJvxvQ1FpiddPXJ49hV3cAj72zxuxRjTIxZYHbS0SP7c+yo/vz5tY3UN1kv05i+xAKzC64+eRzbK+t4fKWdMTemL/EsMEXkPhFZLiI3tPF8oohsEZEl7s8RXtUSbceNGcCU4f24e8kGGm1cpjF9hieBKSLzgYCqzgBGi8i4MM0mAQ+r6mz3530vavGCiHDVSWP5fE8tT71nF0wzpq/wqoc5G1jk3n4JmBmmzXTgDBF52+2NJobbkIgsEJEiESkqLS31ptoumDOhgMMHZ7NwyQaa7bK8xvQJXgVmBhDsepUBhWHavAOcoqrHAEnAF8NtSFXvUdVpqjotPz/fk2K7ItjL/GxXDc+uKfa7HGNMDHgVmNVAmns7s433WaOqJe7tIiDcbnuPNvewgYwvzOSPr66nxXqZxvR6XgXmSvbvhk8GNoVp84CITBaRADAPWO1RLZ5JSBCunDOWT3ZU89KH2/0uxxjjMa8C8yngQhG5FTgXWCsiN4a0+QXwALAKWK6qiz2qxVNnTBrMqLwM7vzPelsv05hezpPAVNVKnBM/K4A5qrpaVW8IafOBqk5S1SNU9Xov6oiFQIJwxewxrC2u5NWPd/pdjjHGQ56Nw1TVPaq6SFV7/b7qvClDGJqbxh2vWC/TmN7MZvpEQVIggctnj2HV1nLeWL/b73KMMR6xwIySrx41lIHZqdzxn0/9LsUY4xELzChJSQxw6YmjefuzMt7aaL1MY3ojC8woOv/o4eRlJnPXq+v9LsUY4wELzChKSw5wyQmjWfbpLt7bssfvcowxUWaBGWUXTB9Bv/Qk7vqP9TKN6W0sMKMsMyWR7xw/ilc+2skH2yr8LqdXefSdLWyvqPO7DNOHWWB64KLjRpKVkshtL39iKxlFye7qeq594n0efWer36WYPswC0wM5aUlcedJYXvloJ9+9/x0qahv9LinuFZc7PcsdVdbDNP6xwPTIZSeO4cZ5E1n26S7O+uMbrN9Z5XdJcW1beS0AOystMI1/LDA99I3pI3jokulU1DYy749vsvjDHX6XFLdKKtzArKr3uRLTl1lgeuyYUf3511UzGZmXziUPFHHnK5/a2pldUOz2MHdYD9P4yAIzBgb3S+Pxy47jK5MH8/uXP+HKh96lpr7J77LiSrF7dnxXdYOdSDO+scCMkdSkALeddyTXf/FQXly7nbPvfpMtu/f6XVbcCPYwm1uU3TW2W278YYEZQyLCJbNGc/+3j6Gkoo4z73qd1z/d5XdZcaGkvI7sVOc6eTsrLTCNPywwfXDCuHye+d7xDMxO5aK/vsW9yzbaOprtaGxuYUdVHUcOzwVgpw0tMj6xwPTJiAEZPHnFccw9bCA3PreO2xbbsnBt2V5RhyocOawfADush2l8YoHpo4yURBZeMJWzpw7lzv98yhvrbfc8nBL3hM/koTmA7ZIb/1hg+iwhQfjlvMMZk5/JNY+uotTGGR4keMJnxIAM+mck22wf4xsLzB4gPTmRu74+hcraRn64aJWN0wxR7A5aH9wvlYKsFOthGt94Fpgicp+ILBeRGzpoVygi73lVR7w4ZGA2/33mYSz7dBf3LNvodzk9SnF5Lf3Sk0hPTqQgO9VO+hjfeBKYIjIfCKjqDGC0iIxrp/ktQJoXdcSbrx8znC8dMYhbXvyYd20B4n1KyusYlOP8Eym0HqbxUbuBKSIJIpLRznPntvHS2cAi9/ZLwMw2tnESUAP0+kvxRkJE+NX8IxiYk8pVD71HxV5b5QichTeG9EsFoCA7hdLqepvtY3zRUQ9zJHCliMwRkfmtf4CzgAvbeF0GsM29XQYUhjYQkWTg58B17RUgIgtEpEhEikpLSzsoN/7lpCVx19ensqOyjmufWGPjM3HOku/rYWan0tyilNU0+FyV6Ys6CswmoBkn2CYDc4EbgfnARKCtb3M1+3ezM9t4n+uAhapa3l4BqnqPqk5T1Wn5+fkdlNs7HDmsHz85bQIvrN3Ogys2+12Or6rrm6iobWRwP+efU0FWCmCLcBh/tBmYIpKIE44zgUHAc8AyYAfwNvBsO9tdyf7d8MnApjBtTsHpvS4BjhSReztZe6/23ZmjmT0hn18+t461xX33Uhcl5fvPkAMUZDt/2vAr44eOepjLgIaQdhryZzhPAReKyK3AucBaEbmxdQNVnaWqs1V1NrBKVb/bqcp7uYQE4ffnTCY3PYmrHnqvz65uFFylKNjDLHQD03qYxg9tBqaqNuGcsHkH53jkncC1wBHA5cDCdl5biXPiZwUwR1VXq2qbw4vc0DQhBmSmcPt5U9i0u4afP/2B3+X4onhfD9MJzPzM4C659TBN7HXUwxyO0/s7SVWPVdVJqpqvqocCxwFJbb1QVfeo6iJVtTPg3TBjzACuOmkcT767jSdWfu53OTFXUl5LgjjDiQCSExPon5FsYzGNLxLbekJEUoCfAXXu8J9QCew/E248dPXJ41ixcTc/f/oDjhzejzH5mX6XFDPbyusozE4lMbD///aCrBTrYRpftLdLXq+qpwM/AkYD84AlwBPuz+PA32JQY58XSBD+cP4UUhITuPIf7+7bTe0LSipqGZSTesBjBdmplFoP0/igw5k+qrpRVc8H/hvYoqor3Z8iVX3D+xINwMCcVP5w/hS2lO3ltNuX8szqYr9Lioni8tp9xy+DrIdp/BLR1EgRKVDVx1X1I/f+ZG/LMuHMGp/P898/gbEFmVz98Ht8/5H3evU1z1WV4oq6gwKz0J3tY4uUmFjrMDBFJAA82ur+AODJDuaHG4+MGJDBoktn8MNTx/PsmhJOv30pb27oneto7q5poKGphcGhu+RZqe61fWy2j4mtSHbJm3Fm/CAiOcBjwP2qakuE+yQxkMDVJ4/jicuPIyUpwAX3vsWv/r2O+qZmv0uLquCx2kFhephgl6owsRfpakUBETkPeBq4DWeWjvHZkcP68dzVM/n6McO5Z+lGvnLXG3y0vdLvsqKmuNwJxCEhgZmf5fQ4bdUiE2sdrVb0bRG5GGcQ+tU4vcv+QI6IfEtELvW+RNOe9OREbjrrCO67eBq7quv58p1vcO+yjb3i+F7ooPUg62Eav3TUw0wHggeQWnCmQ6a6r0sDkr0rzXTGyYcW8sI1s5g1Pp8bn1vHN+57i+0V8R0oJRW1pCQmkJt+4PyI/Cyb7WP80W5gqupdqvpn4A3gYdx54cAeVV2oqnfGoEYTobzMFP5y0VH8ev4RvLelnDPvep3VW9tdDKpHKy6vY0i/NETkgMdTEgPkpifZfHITc5Eew6xV1YXA2Tizf5Z4VpHpFhHha8cM5+nvHU9KYgLn3bOcf79f4ndZXVJcUcugfqlhnyvMTmWnrVhkYizSYUXJAKq6G2fh4DkicrbHtZluGF+YxVNXHs9hg7K54h/v8sdX18fdYsTF5bUMzgl/9ZL8rBR2Wg/TxFikw4q+1up+PXARUCAicz2szXRTXmYKD10yna8cOZibX/yYHz+2Jm6GHjU0tbCzqv6gIUVB1sM0fujoLHm2iFwN/ENE5onIxeIcUNoMVLqPj41FoaZrUpMC3H7ekfzglPE88e7nXHjv23FxeYcdlXWosu9aPqEKslIorbLZPia2Ouph3gGswVlEeDUwDngFOAk4FnhRVdd7WqHpNhHh+6eM446vTWHV5+WctfAN1u+s9rusdrU1pCioMDuVphalbG/PD3/Te3QUmH8AKoAcnGv47AIagQeBgar6DW/LM9H05cmDeWTBdGrqmzhr4Ru8/mnPnVJZ4g6JGtTGMUy7to/xQ0eBeTFwKc5g9aHARmCBqg4EKkXkfI/rM1E2dXguT115PINz0rj4b2/z0Ftb/C4prG0h1/IJFby2jx3HNLHU0TjMa1T1MmCDqt4NnAlcKiI/xrkI2n+JyEGX0DU929DcdB6/fAYnjMvjZ/98n18++2GPOxZYUlFLv/Qk0pPDr3Ed7GHamXITS5GOwwyaCjyJczzzf4ErVXVH1KsynstKTeLei6bxzeNGct/rn/HXNz7zu6QDFJfXtTmkCKAgOD3SZvuYGIo0MDNEZDzO1MjtODN/fgK871VhxnuJgQT+58zDOOXQQn73wsc9auEOZ+Hg8Lvj0Gq2j80nNzEUaWBuxLli5Crgt8CL7p+XugPbu0RE+ovIqSKS19VtmO4REX579hFkpyVxzSOrqGvsGeM0w620HqogK9Xmk5uYiigwVfVbqvodVf2uql6gqicAs2j/Imr3ichyEQl7eV0RycU5DnoM8KqI5HehfhMFAzJTuPmcSXy0vYpbXvzY73Korm+isq6pzTPkQQXZKXbSx8RUZ49h7qOqlar6O3cm0AFEZD4QUNUZwOg2VmefBPxQVW/C6bFO7WotpvvmTCjgohkjuPf1z3wfblTSwRnyoIKsVDvpY2Kqy4HZgdnAIvf2S8DM0Aaq+pqqrhCRWTi9zOXhNiQiC0SkSESKSktLPSrXAPz09EMZk5/Bjx5bRbmPA8KDQ4pCFw4OVZhts31MbHkVmBnsv2Z5GRB26JE7zfI8YA/OgPiDqOo9qjpNVafl59teu5fSkgP84fwplNU08LN/vu/bYh37Bq13eAwzxWb7mJjyKjCrcRYYBshs633UcSXO9Msve1SL6YSJQ3L44akT+Pf723ny3W0dv8ADxeW1JAgUumMt21KYbZeqMLHlVWCuZP9u+GRgU2gDEblWRC5y7/YD4nel215mwazRHDOqP//zzFq2lu2N+fsXl9dRmJ1KYqD9f57BsZg2tMjEileB+RRwoYjcirtKu4jcGNLmHrfNUiCAc6zT9ACBBOHWcycjAj94dBVNzS0xff9IhhSBc9IHoNR6mCZGPAlMVa3EOfGzApijqqtV9YaQNntU9VRVnaWqV2i8rW7byw3NTefGeRMp2ryHP722IabvXVxRy6Cc9s+QQ+tr+1gP08SGVz3MYCAuUtXtXr2H8dZXjhzClycP5vbFn8bs2kAtLUpJRV2HZ8jBWeuzX3qSjcU0MeNZYJre4ZfzJlKQlcIPHl3F3oYmz99vd00DDU0tEfUwAQqzUq2HaWLGAtO0Kyctid+feySf7a7hxufWef5+JRXtLxwcqiA7hR3WwzQxYoFpOjRjzAAWzBrNQ29t4eUPvV2cqqOV1kMVZKVSaj1MEyMWmCYiPzx1PIcNyua/Hl+9byaOF4rLnfDrTA9zp832MTFigWkikpIY4I8XTKW5WbniH+/S0OTNUKPi8lpSkxLITU+KqH2hO9tnj832MTFggWkiNiovg5vPmcTqreXc9NyHnrxHSYWzcLAza7ZjwUtV2DJvJhYsME2nnDZxEJecMIr7l2/mmdXFUd/+tggHrQcVBldet9k+JgYsME2n/eS0Qzh6ZC7XPbGG9Turorrt4vLIBq0HBWf72HxyEwsWmKbTkgIJ3PX1qaQnB7jswXepqY/O+MyGphZKq+s71cMMzvaxHqaJBQtM0yWF2anccf4UNpZW89Mno7MU3I7KOlQ7Xji4tdSkADlpSXYM08SEBabpsuPG5vGjuRN4ZnUxD67Y3O3tdXYMZlBhdorN9jExYYFpuuXyE8dw0iEF/OLZD1nVzfnmxZ2c5RNUmJ1q88lNTFhgmm5JcJeCK8xO5cp/vMuemq6Ph9w3aL2Di5+Fys9KsWv7mJiwwDTd1i89mYUXTKW0qp4fLFrV5Vk3xeW15KYnkZbcuSs3F2anUlpts32M9ywwTVRMGtqP/z7zMJZ8XMpdr67v0jZKKuo6vLRuOAVZKTQ222wf4z0LTBM1Fxw7nLOmDOG2xZ906VK9ka60HmrftX3sOKbxmAWmiRoR4aazJjKuIJOrH3mv02eunVk+kQ8pCiqwlddNjFhgmqhKT05k4QVHUVnbyN1LIr+0RVVdI1V1TdbDND2aBaaJurEFmcybMoRH3tlCWYRnzYPXIu9KYO6b7WM9TOMxC0zjictOHE1dYwv/9+amiNrvG7TeiXnkQcHZPtbDNF6zwDSeGFuQxdzDCrn/zU0RzTXv7MLBoQqybLaP8Z5ngSki94nIchG5oY3nc0TkeRF5SUT+KSLJXtVi/HHZ7DFU1Dby8NtbOmxbUlFLguw/gdNZhdmpNp/ceM6TwBSR+UBAVWcAo0VkXJhmFwC3qupcYDtwmhe1GP9MHZ7LsaP6c++yzzpcoX1beS0Ds1NJDHTtn2RBdgqltktuPOZVD3M2sMi9/RIwM7SBqi5U1Zfdu/nAznAbEpEFIlIkIkWlpaVe1Go8dPnsMWyvrOOpVdvabVdcXsugLu6Og7Mu5s6quqismmRMW7wKzAwg+A0pAwrbaigiM4BcVV0R7nlVvUdVp6nqtPz8/OhXajx14vh8Dh2UzZ9f29Du1MWSirouH78EZ8UiZ7ZPY5e3YUxHvArMaiD4rz+zrfcRkf7AncC3ParD+ExEuHz2GDaU1vDyuvCX6G1pUUrK67p0hjwouPK6nfgxXvIqMFeyfzd8MrAptIF7kucx4Keq2v3FFE2P9cWJAxneP52FSzaE3WXeXdNAQ3NLt3uYYIPXjbe8CsyngAtF5FbgXGCtiNwY0uY7wFTgehFZIiLneVSL8VliIIFLZo1m9dZyVmwsO+j5ri4c3Jr1ME0seBKYqlqJc+JnBTBHVVer6g0hbe5W1VxVne3+POpFLaZnOOeooeRlJnP3awdPlyxxFw7uzMXPQhW4PUw7U2685Nk4TFXdo6qLVHW7V+9h4kdqUoBvHT+KpZ+Usra44oDntrmD1od0o4eZmhQgOzXRepjGUzbTx8TMN6aPIDMlkT+9tvGAx0vKa0lNSqBfelK3tl+YnWqX2zWessA0MZOTlsQF04fz3JpiNu+u2fd4cYWzDqaIdGv7Bdkp7LDL7RoPWWCamPrO8aNITEjgnqX7e5nbyus6fR2fcAqzrIdpvGWBaWKqIDuVs48awmMrP2en2xss6eLCweG2bbN9jJcsME3MLZg1hsbmFv72xiYamloora7v0rV8Qu2/to/N9jHesMA0MTcqL4MvThzEg8s38+nOKlS7d4Y8aP/K63YrfkRxAAANWElEQVQc03jDAtP44rITx1BV38QtL34MdG/QelBwLKYt82a8YoFpfHHE0BxOGJfHqx87K1ANisIxzEJ3to9dqsJ4xQLT+OayE8fsux2Ns+QFNp/ceMwC0/jmuDEDmDQ0hwEZyaQlB7q9veBsH+thGq8k+l2A6btEhDvOn0KxO5c8GgrsUhXGQxaYxlcj8zIYmZcRte0VZqfYWXLjGdslN71KQZb1MI13LDBNrxK8GJrN9jFesMA0vUphVioNzS2U22wf4wELTNOr7Bu8bscxjQcsME2vsm96pB3HNB6wwDS9SkFWcHqk9TBN9Flgml4leDE0m+1jvGCBaXqVtOQAWTbbx3jEs8AUkftEZLmI3NBOm0IRWeZVDaZvKsxOtR6m8YQngSki84GAqs4ARovIuDBtcoH7gehN8zAG5zimHcM0XvCqhzkbWOTefgmYGaZNM3AeUOlRDaaPsh6m8YpXgZkBbHNvlwGFoQ1UtVJVK0IfDyUiC0SkSESKSktLo1ym6Y0KslLYWVnPu1v22IwfE1VeBWY1EFzgMLM776Oq96jqNFWdlp+fH5XiTO829/BCUhITmL/wTc6483UeeXsLtQ3NfpdlegGvAnMl+3fDJwObPHofYw5y1Ij+rPjZydx01kSaW5TrnnyfY3+1mF/860M2llb7XZ6JY+LFLouIZAPLgFeA04HzgXNU9aAz5iKyRFVnR7LdadOmaVFRUTRLNb2cqlK0eQ9/X76ZFz4oobFZOWFcHt+YPoKTDykgMWAj6wyIyEpVndZhO6+O8bhnwU8Flqrq9mhs0wLTdMfOqjoefXsrD729hZKKOgbnpPL1Y4czf+rQqFyEzcQv3wPTCxaYJhqamltYvG4nD67YzOvrdwFw6KBsTjm0gJMPLWTSkBwSEsTnKk0sWWAaE4HPdtXw0trtvLJuJ0Wby2hRyM9K4aQJBZx8aAEzx+WRnmwXJujtLDCN6aQ9NQ0s+WQni9ftZOnHpVTVN5GcmMBxYwZw8qGFnHxIge2691IWmMZ0Q2NzC+98VsbidTt55aMdbN69F4DR+RnMGD2A48bkMX10fwZkpvhcqYkGC0xjokRV2VBazasflbJ8427e2ribGndc5yEDszhuTB7HjRnAMaP7k52a5HO1pissMI3xSGNzC+9vq2D5ht28uWEXRZv2UN/UQoLAEUNymDEmj6NH5jJxSA4FWSmI2Amkns4C05gYqWtsZtXWct7csJvlG3bx3pZymlqc71VeZjKHD85h4pBsJg7OYeKQHIbmplmI9jCRBqad/jOmm1KTAkwfPYDpowfAqePZ29DE2uJK1m6r4IPiSj7YVsHr63fR7IZodmrivhA9fHAOYwsyGZOfSVpywOdPYjpigWlMlKUnJ3L0yP4cPbL/vsfqGpv5ZEcVH2yr5IPiCtYWV3L/8s00NLXsazOkX9q+8HT+zGBsQaadWOpBLDCNiYHUpACThvZj0tB++x5rbG5hY2kNG0qrWb/T+dlQWs1bn+2mrnF/kOamJzEmP5NReRkM75/OsP7pDOufxrDcdPLtGGlMWWAa45OkQAITBmYxYWDWAY+3tCjFFbVugNbsC9Iln5RSGrLOZ2pSAkNz0xmWm8aw/ukM75/O0Nx0hvRLY2BOKgMykm3WUhRZYBrTwyQkCENzneCbPeHA5+oam/l8z162ltWypWwvW8v2snXPXraU1VK0aQ9V9U0HtE8OJFCYk8KgbCdAB+Wkun+m7bs9ICPZFiGJkAWmMXEkNSnA2IIsxhZkHfScqlJR28iWsr2UVNSxvaLO/bOW4oo6Vm0t54UP6mhobjngdSIwICOZvMwU8rNa/bS6X5CVwoCMFLLTkgj04R6rBaYxvYSI0C89mX7pyUwaGr6NqlJW07A/UCvr2FVVT2l1PaVVzs/G0hpKq+sPOCG1/z2gX1oSuRnJ9E9PJjcjmdz00PvJ9EtPIictiX5pSWSnJZGa1DtGAFhgGtOHiAgDMlMYkJnCxCE5bbZTVSrrmvaF6M6qOvbUNFC2t9H9s4E9NQ1sLdvL+583UlbTcFDPtbXUpAQ3QJPJSUsixw3U7NQkslITyUpNbHV7/2PB2z0lcC0wjTEHEREn2NKSGFuQ2WF7VWVvQzNlNQ3s2dtARW0j5Xsbqajd/1Pe6vGtZXv5oLaRytrGfdNM25McSCAjJUBGSiIZyYkht/ffz0xJpCArhXOmDYvGX8NBLDCNMd0mIm5wJTKsf3qnXtvcolTXNVFZ10hVXRPV9U1Uuber6hqprGuiqq6JmvomahrcP+ubqaprYkdlHTX1zVTXO483tShjCzItMI0xvVMgQZxd9PTuL1xS39R8wBjWaLPANMb0GimJAVISvTveaYOvjDEmQhaYxhgTIQtMY4yJkGeBKSL3ichyETnoWuSdaWOMMT2FJ4EpIvOBgKrOAEaLyLiutDHGmJ7Eqx7mbGCRe/slYGYX2yAiC0SkSESKSktLo1ymMcZEzqvAzAC2ubfLgMIutkFV71HVaao6LT8/P+qFGmNMpLwah1kNBC/gnEn4YI6kzQFWrly5S0Q2d6KOPGBXJ9rHA/tM8cE+U3wIfqYRkTT2KjBX4uxirwAmAx93sc0BVLVTXUwRKYrkwkbxxD5TfLDPFB86+5m8CsyngGUiMhg4HThfRG5U1RvaaTPdo1qMMSYqPDmGqaqVOCd1VgBzVHV1SFiGa1PhRS3GGBMtns0lV9U97D8L3uU23XSPh9v2i32m+GCfKT506jOJqnpViDHG9Co2NdIYYyJkgWmMMRHqtYHZ2+api0iiiGwRkSXuzxF+19QdIlIoIsvc20ki8i8ReUNEvu13bV0V8pmGiMjnrX5fcTXrQkRyROR5EXlJRP4pIsnx/p1q4zN16jvVKwOzl85TnwQ8rKqz3Z/3/S6oq0QkF7gfZ7YXwFXASlU9HviqiBx8DdkeLsxnOha4qdXvK97m9V4A3Kqqc4HtwPnE/3cq9DNdRye/U70yMIlwnnqcmQ6cISJvu//Tx/Nq+c3AeUCle382+39fS4F4HBwd+pmmA98VkXdF5Ff+ldU1qrpQVV927+YD3yDOv1NhPlMTnfxO9dbAjGieepx5BzhFVY8BkoAv+lxPl6lqZci427j/fYX5TM/j/EdwNDBDRCb5Ulg3icgMIBfYSpz/joJafaaX6eR3qrcGZqfnqceBNapa4t4uAuJxl6gtvfH39aaqVqlqM/Aecfj7EpH+wJ3At+klv6OQz9Tp71RcfugIBOepgzNPfZN/pUTNAyIyWUQCwDxgtd8FRVFv/H29KCKDRCQdmAt84HdBnSEiycBjwE9VdTO94HcU5jN1+jsVz8fB2tMb56n/AngIEOAZVV3scz3RdD/wbxE5ATgMeMvneqLhf4FXgQbgT6ra4eIyPcx3gKnA9SJyPfA34MI4/06FfqZXgQfoxHeq1870cc9angosVdXtftdj2ud+EWcCL9q6Aj2Tfad6cWAaY0y09dZjmMYYE3UWmMYYEyELTOMrcYV7vJPbGRmtmtztjQ83VU5ERkfzfUx8scA0MSUiv3XnjgdE5A6cKYR3hGl6k4h8UUQyROQpEckSkbltbPNanLOf3alrmIh8v9VD1wCjwjS9XERmdee9TPyywDSxlgh8FWcWzDnAXcCX3cUPbmrV7iRgiarWACOBRuB3IjKs9cbcnuUwVX3Svb9vAQz3/kELe7Sx2Mc24HQRmSsiSW6NP2q1MMPdbrufAleLiH13+iD7pZtY+5mqPowzZfBonJ7c3cAcnLGmiMipQLGq7nVf06SqdThT10aGbO9C4I/u60IXwIDwC3sc9JiqtuCM0/sK8C3gtziheSpwC7AbQFWbgFeA46Pyt2HiigWmiRkR+Q6wRES+CpwB/B2nh3kxzrzexSIyGbgJaBCRY0Xke8AwEXkc+A1wTMhmx6jqOvd26AIYEH5hj3CPoarbVPVKYIpb1y9xgnEw8ESrba5w25g+prfO9DE9kKreJyL1QKaqzgQQkfuBSlW9yr1/Bc7CFROAITiXX34HuFZVN3Sw/Up3G60fDrewR9jFPtzVav6iqt9y7xcAKaoaet2XWvbPqzZ9iPUwjW9E5FigAChttQTaX3AWR0BVn3SX43oW+EIbm6kVkcx23ibcohFtLSRxMbDBrS0TGAOcFWabo3BW7zF9jAWm8YV7nPI+4FJV/QUwVkR+rKqNYZo/AXxTRBLCDDf6N3B2O28VbtGIgx4TkUHAAuBm9/GbgZ8De91DCa3Nw1kT0vQxtktuYsY9s3wmzq72YuBsVd3iPv1d4ORgU/cHAFUtFZF/AX8AdojIx6r6mPv0s8CjIvK8qu4M87bhFvbYFuaxHwC/BoaLyG3Aa6r6jIg8DywSkenADThLgJWq6q5o/J2Y+GJzyU3MiMilOLuzi4ErgeE4w4WSgFSgH85Z8+XAXao6L+T1/w9n5e9Zqlrc6vGh7mMPtfG+By3s0dZiHyJyMbCt9co1bq/2CuBp4FzgzjZ6wqaXs8A0MSMiCe7wHWPikgWmMcZEyE76GGNMhCwwjTEmQhaYxhgTIQtMY4yJkAWmMcZE6P8DEr+kR0UwCiYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "parameters = L_layer_model(train_x, train_y, [train_x.shape[0],7,1], num_iterations = 2500, print_cost = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率:0.9999999999999998\n"
     ]
    }
   ],
   "source": [
    "pred_train=predict(train_x,train_y,parameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率:0.74\n"
     ]
    }
   ],
   "source": [
    "pred_test=predict(test_x,test_y,parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四层神经网络的应用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost after iteration 0: 0.771749\n",
      "Cost after iteration 100: 0.672053\n",
      "Cost after iteration 200: 0.648263\n",
      "Cost after iteration 300: 0.611507\n",
      "Cost after iteration 400: 0.567047\n",
      "Cost after iteration 500: 0.540138\n",
      "Cost after iteration 600: 0.527930\n",
      "Cost after iteration 700: 0.465477\n",
      "Cost after iteration 800: 0.369126\n",
      "Cost after iteration 900: 0.391747\n",
      "Cost after iteration 1000: 0.315187\n",
      "Cost after iteration 1100: 0.272700\n",
      "Cost after iteration 1200: 0.237419\n",
      "Cost after iteration 1300: 0.199601\n",
      "Cost after iteration 1400: 0.189263\n",
      "Cost after iteration 1500: 0.161189\n",
      "Cost after iteration 1600: 0.148214\n",
      "Cost after iteration 1700: 0.137775\n",
      "Cost after iteration 1800: 0.129740\n",
      "Cost after iteration 1900: 0.121225\n",
      "Cost after iteration 2000: 0.113821\n",
      "Cost after iteration 2100: 0.107839\n",
      "Cost after iteration 2200: 0.102855\n",
      "Cost after iteration 2300: 0.100897\n",
      "Cost after iteration 2400: 0.092878\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUwAAAETCAYAAABdtj99AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VOXZ//HPNZOVBEiAECBhN6DsS1yooKEK1hVEtPax2GoVFNfWLrbSn60PdlEf2qqFgtJatdoiKi64LywiLgk7yiYCEgQChEDYsl2/P86hxDBJJjAnJ5Nc79crL2a558w1hPly3+ec+z6iqhhjjKldwO8CjDEmWlhgGmNMmCwwjTEmTBaYxhgTJgtMY4wJkwWmiQgR6ez+mSoit1V6/HwR+UUE3+deEfl+pLZnTF1YYJqTJiJZwCci8i3gCDBRRH7oPn0PEFul/SoR2SQia0VkTZWf3SIytFLbmSKyQETmicg84FrgwaP3RWShiLwYRo03ich2EckVka61tA2KyGMiUiAiz4lIvPt4kojMEZGdIjK1UtvtVX4q3Oe+JyL7Kj1+Z5h/paaBivG7ABP9VHW9iNwIdFbVD90eYJGIXAgIcH+V9n1EpJmqHqy6LRHJBfZVeuh+4KCqbheRHsDjwAggoKqHRKQ3UF5TfSLSF/g1MAjoDDwKXFzDS24CugAZwF3AHcADwG+AQqAdMFVErlLVWe79o+810n09QH9ggqo+W1N9JnpYD9OcFBGJE5Fy4D7gFyKyDJgJzAZ+D6QAS0WkXERi3dc0A1aLyH0iklBlk/FA5SA9G3hDRLoBfwZ+BYwHnnB7trOAge52XwrR2/sdMBp4UlW3qepioI2IJNXwscYAD6lqCTAVGFXp8T+oaoX7+OgQr73H/bvArWtZDe9jooz1ME0k7MfpvcUCJRpi+piI7D16W1UPisgQnNBZJiIjVXWL+3Q8cLhS26dEZB0wDOgA/B9QAHwXOBUYr6qL3LZHg63qe08H5lV6aBtOT/Ozaj5PJrDC3WaRiKS5j7cB1rm3NwOnVHmfQUC5qh4NyYHAC+7r3wWuC9WrNtHDepjmZJUC/YDBQB6Q6+4nzBORIyKi7gGhbqpaevRFqrpdVcfgDHO3VdpeHJUCU0Sa44TjWpwe5jKcHub7QALwB3fIXZMg3xzmH8Dp+Ybb/mjH4mCl/wxCbWMCMN2tOwl4kWNBHwtMrKVO08BZYJqToo4tqvqpqvZR1cHADThB+mucoNysqnuq2cQc4BoRecm9H49z4AgRCQILgTVAFs7+xAHAE8CXwOXAz9z3q2lIXsg3wy0RqKjhY1Vtf3S3QYWISKhtiEgccAnwsvv3ckBVJ6jqLndo/zjw7Rre00QBG5Kbk+IecV6I0+M62vtqBqTihM71bsY0A85V1S/d1/UGfojTe3wP+KP72spDcgGGq2qhe/8pEZkMbMfpyVWoarnbm42pYUj+XZye3r/cwBsE5NfwsXKBIcBs9/MdHUavwOlJ5wLZwNZKrzkP+FhVD7nv2Rnooapvu89nUHNImyhggWlOihuAmZUfE5Ec4E5VPe6giIi0BuYDrYG/AP1UdW+lJsFKQ/eLgbtF5Eil57sCJcBYd3vgDKH/Cvy7mjJfB6aIyHtAH2C3qn5Vw8d6FpguIgXAnTi9YIBngD+LyN04R++fqfSaC3GCv7JpInK0V3krztF5E8UsME1EuYHYiWO9zW9Q1d3uie0fqurRoXc8zj6+/rjDcbftS8BLlV9/tIepqmGHj6ruE5GxwIPu9q9xt/Ut4B5VvbhK+wUi8hBOoK8Afus+/pSIZALTcMLxb5Vedi7wZKVtbBaRB4GlOD3Uv+OcPWCimNh6mCaS3JPIs3BOv3k6zNcMwBnmbgX+rKp/rqHtH3EC80+RqNeYurDANA2CiATc8xuNabAsMI0xJkx2WpExxoTJAtMYY8LkWWC6q8wsFpFJ1TyfKiKvubNCpntVhzHGRIonpxWJyBic8+mGiMjfRSRLVddXaTYO+Jeq/ktEnhGRbFXNrWm7bdq00S5dunhRsjGmCcvLy9ulqmm1tfPqPMwcnFVkAN4ChgJVA3M30EdEUoCOQMgTiUVkPM7cYTp16kRubo2ZaowxdSYim8Np59WQPIljU8/2AOkh2nyAs2LM7cDnbrvjqOoMVc1W1ey0tFr/AzDGGM94FZjFOIsTACRX8z73Ajep6n04iytc51EtxhgTEV4FZh7OMByc6W6bQrRJBfq6K9KcSTVT6YwxpqHwKjDnAONEZApwFc7q2pOrtPk9MAMoAlrhLHhgjDENlicHfdzFDnJwrr3ygKpuB5ZXafMJ0NuL9zfGGC94tlqRu4bhrFobGmNMlGjUM33W7diPzZU3xkRKow3MV1dsY+SfFrBia5HfpRhjGolGG5jDstKIiwnw4tKarkRgjDHha7SB2TIxlhG90nl5+TZKy22ZRWPMyWu0gQkwZmAGew6UMH9tgd+lGGMagUYdmOf0SKN1UpwNy40xEdGoAzM2GODS/h14+/MdFB0qrf0FxhhTg0YdmACXD8ygpKyC11d+7Xcpxpgo1+gDs19mS7qnJfHCEhuWG2NOTqMPTBFhzKBMPtm0h6/2HPS7HGNMFGv0gQkwakAHAObYwR9jzEloEoGZmdqMM7u24sWl+TZV0hhzwppEYAKMGZTBxl0HWPbVXr9LMcZEqSYTmBf2bU+8TZU0xpyEJhOYLRKcqZKvLN9GSZlNlTTG1F2TCUxwhuWFB0uZv86mShpj6q5JBeawrKNTJbf6XYoxJgo1qcA8OlXync92UnTQpkoaY+qmSQUmOMPykvIK5tpUSWNMHTW5wOyb0ZJT2ibbsNwYU2eeBaaIzBSRxSIyqZrnbxaRee7PMhGZ7lUtVd6Xywdm8OmmQpsqaYypE08CU0TGAEFVHQJ0E5Gsqm1UdZqq5qhqDrAQeMyLWkIZPTADwM7JNMbUiVc9zByOXWL3LWBodQ1FJANIV9Vcj2o5TkZKImd1a8ULS7baVEljTNi8Cswk4Gj3bQ+QXkPbW4Bp1T0pIuNFJFdEcgsKInf+5JiBmWzafZClNlXSGBMmrwKzGEh0bydX9z4iEgCGA/Oq25CqzlDVbFXNTktLi1iBF/Zt50yVtHUyjTFh8iow8zg2DO8PbKqm3TDgY/VhXNw8IZaRvdvxygqbKmmMCY9XgTkHGCciU4CrgNUiMjlEuwuABR7VUKsxAzPYe7CU99fu9KsEY0wU8SQwVXUfzoGfj4DhqrpcVY87vUhVf6WqL3hRQziGZbWhTXKcDcuNMWHx7DxMVS1U1Vmqut2r9zhZMe5UyffW2FRJY0ztmtxMn6quGJRJSXkFr67c5ncpxpgGrskHZu8OLeiRnszD765ngS37ZoypQZMPTBFhylUDSI6P4dq/f8LPZy+n6JANz40xx2vygQnQJ6Mlc28fxsSc7jy/JJ8RU+bz9mc7/C7LGNPAWGC6EmKD/Pw7pzJn4tm0Sorjxidzuf3Zpew5UOJ3acaYBsICs4q+mS15+dah/Pj8Hry+6mtGTJnPqyu22ZxzY4wFZihxMQHuOD+LV24bSkZqIrc+s5Sbns5j577DfpdmjPGRBWYNTm3Xghdu/hZ3X3gq768t4Pwp85mdZyscGdNUWWDWIiYY4KZzu/P6HcPokd6cnz63nPte/cxC05gmyAIzTN3Tkpk1YQg//FYX/rFoE/fP/dxC05gmJsbvAqJJICDce2kvVJXHP/iSYFC4+zunIiJ+l2aMqQcWmHUkIvzmst6UqzJ9/kaCIvzsgp4WmsY0ARaYJ0BEuO+yPpRXKFPnfUEwIPxkRA8LTWMaOQvMExQICPeP7ktFBTzy3gYCIvx4RA+/yzLGeMgC8yQEAsLvx/SlXJW/vLueYEC4/bzjLpBpjGkkLDBPUiAg/PGKflRUKFPeXkcwINwy/BS/yzLGeMACMwKCAeHBK/tTrsqDb64lGBBuOre732UZYyLMAjNCggHh/67sT4XCH15fQ1CEG8/p5ndZxpgIssCMoJhggD9d1Z+KCuX+1z4nEBB+NLSr32UZYyLEAjPCYoIB/nz1AMorlP999TM6piYysnc7v8syxkSAZ1MjRWSmiCwWkeOuFlml3VQRudSrOvwQ64Zmv8yW/Pg/y1i7fb/fJRljIsCTwBSRMUBQVYcA3UQk5Lk2IjIMaKeqr3hRh58SYoPMGJdNs/gYbnwyl0JbiNiYqOdVDzMHmOXefgsYWrWBiMQCjwGbRGSUR3X4ql3LBKaPG8z2osPc+uwSysor/C7JGHMSvArMJCDfvb0HSA/R5lrgM+AB4AwRuS3UhkRkvIjkikhuQUH0XdVxUKdU7r+8D4s27Gby3M/9LscYcxK8CsxiING9nVzN+wwEZqjqduBpYHioDanqDFXNVtXstLQ0T4r12pXZHbn+7K488eEmZn36ld/lGGNOkFeBmcexYXh/YFOINhuAoycqZgObPaqlQfjVRacyLKsN98xZSd7mPX6XY4w5AV4F5hxgnIhMAa4CVovI5CptZgLDRWQBMBF4yKNaGoSYYIBHvjeQDimJTHhqCdv2HvK7JGNMHXkSmKq6D+fAz0fAcFVdrqqTqrTZr6pXquo5qjpEVfNDbasxSWkWx+PXZnO4tJwJT+VxuLTc75KMMXXg2XmYqlqoqrPcfZTGlZXenD9/dwCrthXx89kr7DIXxkQRu6aPD87vlc5PR/bk5eXb+Nv8jX6XY4wJkwWmTybmdOeSfu154M01vLdmh9/lGGPCYIHpExHhwbH96dW+BXc8u4zPtu3zuyRjTC0sMH2UGBdkxrXZJMYFGTNtEbM+/cr2aRrTgFlg+iwjJZFXbx/K4M6p/Pz5Fdz5n2UUHynzuyxjTAgWmA1A2+YJPHn9mfx0ZA9eWb6NSx/5gFX5RX6XZYypwgKzgQgGhFu/ncWzN57FoZJyxkz9kCcXb7IhujENiAVmA3Nmt9a8dscwhma14f+9tJqbn15C0aFSv8syxmCB2SC1SnJmBN1z0Wm88/kOLn54IUu3FPpdljFNngVmAxUIOBdRe+6mIQBc+bfFzFjwBRUVNkQ3xi8WmA3cwE6pzL19GCN6pfO719Yw7u8f88aq7TYP3RgfSDQdVMjOztbc3Fy/y/CFqvL0R5v50zvr2XOghKS4ICN6pXNxvw4My2pDQmzQ7xKNiVoikqeq2bW2s8CMLmXlFSzeuJu5K77mjdXb2XuwlObxMW54tmdoVhviYyw8jakLC8wmoLS8gg+/2M3cFdt4c/UOig6V0jwhhpG92nFJv/YMy2pDTND2uhhTGwvMJqakrIJFX+xi7oqveXP1dvYfLmP0gA78+eqBfpdmTIMXbmBa96ORiIsJMLxnWx66sj+5k85n/DndmLNsGx9v3O13acY0GhaYjVB8TJAfn9+DjJREfvPKZ5TbqUjGRIQFZiOVGBfkVxedxudf7+Pfn27xuxxjGgULzEbsor7tOLNrKx56cy1FB216pTEnywKzERMR7r20N0WHSvnTO+v8LseYqOdZYIrITBFZLCKTqnk+RkS2iMg896evV7U0Zb06tOB7Z3TiqY82s37Hfr/LMSaqeRKYIjIGCKrqEKCbiGSFaNYPeFZVc9yflV7UYuCukT1Jigty36uf2XJxxpwEr3qYOcAs9/ZbwNAQbc4CLhGRT9zeaIxHtTR5rZLi+MmIHixcv4u3P7MLrhlzorwKzCQg3729B0gP0eZT4HxVPQOIBS4KtSERGS8iuSKSW1BQ4EmxTcE1Z3Umq20yk+d+bgt3GHOCvArMYiDRvZ1czfusUNWv3du5QKhhO6o6Q1WzVTU7LS0t8pU2EbHBAPde2pstew4y84Mv/S7HmKjkVWDmcWwY3h/YFKLNUyLSX0SCwGhguUe1GNfQrDaM7JXOX9/fwPaiw36XY0zU8Sow5wDjRGQKcBWwWkQmV2lzH/AUsAxYrKrveFSLqWTSxb0oq1AeeGON36UYE3U8CUxV3Ydz4OcjYLiqLlfVSVXarFLVfqraV1Xv8aIOc7xOrZtx47CuvLA0nyV22Qtj6sSz8zBVtVBVZ6nqdq/ew5yYiTmnkN4int++vNoueWFMHdhMnyYoKT6Guy88leVbi5i9ZKvf5RgTNWoMTBEJiEhSDc9d5U1ZxmujB2QwqFMKD7yxlv2HbZ65MeGorYfZBbhFRIaLyJjKP8DlwDjPKzSeEBF+c1lvdhUf4dH3NvhdjjFRobbALAPKgV/jnB40EpgMjAH6ALYDLIr1y0zhysGZ/H3Rl2wtPOh3OcY0eNUGpjtVcTLO+ZTtgbnAQmAH8Anwan0UaLx154gelFUos/NsX6Yxtamth7kQKKnSTqv8aaJYRkoiZ3dvw+y8rXbE3JhaVBuYqlqGs3DGpzjzwh8BfgH0BW4GptZHgcZ7YwdnsrXwEB9/ucfvUoxp0GrrYXYClqnqt1X1TPdE8zRVPQ34Fs6iGSbKXdC7Hc3jY3gu7yu/SzGmQat2STURiQd+BRwWkW+HaBLg2IpEJoolxgW5pH975izdxn2jykiOt5X2jAmlpiH5EVW9ELgL6IazQMY84Hn3Zzbwj3qo0dSDsYM7cqi0nNdWfl17Y2OaqFq7Eqq6EbhaRMYCW1TVVm1ohAZ1SqFbmyRm527lquyOfpdjTIMU1tRIEWmrqrOPhqWI9Pe2LFPfRIQrBmfyyaY9bNp1wO9yjGmQag1Md73K/1S63xp4oZrr9JgodsWgTAICz9v8cmNCqjUwVbUcZ8YPItISeA74p6qu97g2U8/atUxgaFYaz9s5mcaEFO5qRUER+S7wEvAn4HzvSjJ+unJwJtuKDvPhF7v9LsWYBqfGgz4icj3OXPIcIB54BmgFtBSR64A4VZ3udZGm/ozolU6LhBhm533F0Kw2fpdjTINSWw+zGZDg3q7AmQ6Z4L4uEYjzrjTjh4TYIJcN6MAbq7ezz5Z9M+YbagxMVX3U7UEuAp7FvT4PUKiqU1X1kXqo0dSzsYM7cri0grkr7JxMYyoLdx/mIVWdClyBM/tnnmcVGd/1z2xJVttkW8HImCrCPa0oDkBVd+MsHDxcRK7wuDbjExFh7OBM8jYX8kVBsd/lGNNghHta0fcq3T8CXAu0FZGR1b1ORGaKyGIRmVRdG7dduogsrUPNph5cPjCDYEB43nqZxvxXbdf0aSEitwP/EpHRIvIDERFgM7DPffyUEK8bAwRVdQjQrZaT3B/COYBkGpC2LRI4t0caLyzJp9zOyTQGqL2H+TCwAmcR4eVAFvAu8G3gTOBNVQ11QZgcYJZ7+y2cVduP466CdACwS/E2QGMHZ7J932E+2LDL71KMaRBqC8y/AEVAS5xr+OwCSoGngXaq+v1qXpfEsaXf9gDpVRuISBzOtYLurqkAERkvIrkikltQUFBLuSaSzjutLSnNYnku19bJNAZqD8wfABNwTlbPBDYC41W1HbBPRK6u5nXFHBtmJ1fzPncDU1V1b00FqOoMVc1W1ey0tLRayjWRFB8TZFT/Drz12Q6KDoZ/TmbxkTJm523lUEm5h9UZU/9qOw/zTlW9CfhCVacBlwITROSnOBdB+5mIHNd7BPI4NgzvD2wK0eZ8nEv4zgMGiMjjJ/YRjJeuzO5ISVkFr6zYFlb7N1dvZ8SU+fz0ueVMm/+Fx9UZU7/CPQ/zqEHACzj7M38L3KKqO0K0mwOME5EpuCe7i8jkyg1U9RxVzVHVHJzLYNxQ5+qN53p3aMGp7ZrzXC1Hy/P3HuKGf+Yy4ak8WibGcnqXVJ5Y9CXFR8rqqVJjvBduYCaJSA+cqZHbcWb+/BxYGaqxqu7DOfDzETBcVZerarWnF7mhaRqgo+dkLv9qL+t37D/u+bLyCh5fuJERU+azaMMufnnhqbxy21AmXdyLfYfLeObjzT5UbYw3wg3MjThXjFwG/BF40/1zgnti+3FUtVBVZ6mqHQGPcqMHZhATkONm/iz/ai+XPbqIyXM/58yurXjrx+cw4dzuxAYD9O+YwtmntObxhV9yuNT2ZZrGIazAVNXrVPVHqnqDql6jqsOAcwjjEhcm+rVJjienZ1teWJpPWXkF+w+Xcu9Lqxg9dRG7io8w9ZpB/P2Hp9OxVbNvvO6WnFPYuf+ILUhsGo0TDjx32P1ABGsxDdiV2Zm88/kOfv/6Gl5dsY2d+49w7VmdueuCnrRICH215SHdW9O/YwrT52/ku9kdiQnWdZe5MQ2L/Qs2YRnesy2tkuKY+cGXtE6K58WJZ/PbUX2qDUtw9n9OzOnOlj0HmWtXozSNgA2pTVjiYgI8cEU/tu87zNWnh99bHHFaOlltk5k27wsu698BZ2atMdHJepgmbOf3Suf7Z3Wu09A6EBBuzunOmu37eW/NTg+rM8Z7FpjGc5f270BGSiJ/fX8DqraQh4leFpjGc7HBADed240lW/by8Zd7/C7HmBNmgWnqxZXZHWmTHMfUeTZd0kQvC0xTLxJig1w/tCsL1hWwcmuR3+UYc0IsME29+f5ZnWmeEMO0+aGWUDWm4bPANPWmRUIs1w7pzOurtrNhp10ryEQfC0xTr647uytxwQDTbek3E4UsME29apMcz/fO6MSLS/PJ33vI73KMqRMLTFPvbjynGwCPLdjocyXG1I0Fpql3GSmJjBqQwb8/3cLu4iN+l2NM2CwwjS9uzunGkbIKnvhwk9+lGBM2C0zji1PaNueCXu144sNN7D8c/gXWjPGTBabxzcTh3dl/uIx/fbzF71KMCYsFpvFNv8wUhmW1Ydq8L9i064Df5RhTKwtM46vJo/sQELj+iU/Ze7DE73KMqZGvgSkirURkhIi08bMO45/OrZOYcW02WwsPMeGpPErKKvwuyZhqeRaYIjJTRBaLSMjL64pIKvAqcAbwvoikeVWLadhO79KKB8b24+Mv9/DLF1bampmmwfIkMEVkDBBU1SFANxHJCtGsH/ATVb0f57K9g7yoxUSH0QMzuPP8LJ5fspW/vm+Lc5iGyaseZg4wy739FjC0agNVna+qH4nIOTi9zMWhNiQi40UkV0RyCwoKPCrXNAR3nJfF5QMzeOitdbyyfJvf5RhzHK8CMwnId2/vAdJDNRLniljfBQqBkCfjqeoMVc1W1ey0NBu1N2Yiwh+u6MvpXVK567nl5G0u9LskY77Bq8AsBhLd28nVvY86bgFWAJd5VIuJIvExQaaPy6Z9ywTGP5nLlt0H/S7JmP/yKjDzODYM7w9sqtpARH4hIte6d1OAvR7VYqJMq6Q4/vHD0ymrUK574hOKDtlMINMweBWYc4BxIjIFuApYLSKTq7SZ4bZZAARx9nUaA0C3tGSmjxvMlj0HmfivPErL7XQj4z9PAlNV9+Ec+PkIGK6qy1V1UpU2hao6QlXPUdWJaueSmCrO6taa34/px6INu/n1nFV2upHxXYxXG1bVQo4dKTfmhIwdnMmmXQd49P0NdGmTxE3ndve7JNOEeRaYxkTKT0b0YNPuA/zh9TW0b5nAqAEZfpdkmigLTNPgBQLCQ1f2p2D/EX78n2WUlitjB2f6XZZpgmzxDRMVEmKDPHHdGXyrext+Nns5//7EloQz9c8C00SNxLggj/8gm3N7pHH3Cyt5cvEmv0syTYwFpokqCbFBpo8bzIhe6fy/l1bz+EK7kJqpPxaYJurExwSZes0gLurbjslzP7fFOky9sYM+JirFBgM8fPVAYoPLefDNtZSUVXDn+Vk4yxMY4w0LTBO1YoIBplw1gNhggL+8u56S8gp+fkFPC03jGQtME9WCAeGBK/oRFxNg2rwvKCmrYNLFp1loGk9YYJqoFwgI94/uQ1wwwMwPvqS0vILfXNqbQMBC00SWBaZpFESEey/tRXxMgOkLNlJaXsH9o/taaJqIssA0jYaIcPeFpxIbDPDo+xuICQS4b1RvG56biLHANI2KiHDXyB6UVlQwff5GEmID/Ooi26dpIsMC0zQ6IsLd3zmVwyXlPLbwSxLjYvjJiB5+l2UaAQtM0yg5+zR7c6i0nIffXU9ibJCbc2xpOHNyLDBNoxUICL8f04/DpRX88Y01JMYG+OHZXf0uy0QxC0zTqAUDwv9d1Z8jZeX85pXPSIgNcvUZnfwuy0Qpm0tuGr3YYICHvzeQc3uk8csXVzJnaX7tLzImBAtM0yQ4l+8dzFldW3PXc8t5Y9XXfpdkopAFpmkyEmKd9TT7Z7bktmeX8v6anX6XZKKMZ4EpIjNFZLGITKrm+ZYi8rqIvCUiL4pInFe1GHNUUnwMT1x/Bj3bNWfC03l8uGGX3yWZKOJJYIrIGCCoqkOAbiKSFaLZNcAUVR0JbAe+40UtxlTVIiGWp64/k66tk/jRP3P5z6db2FV8xO+yTBTw6ih5DscusfsWMBRYX7mBqk6tdDcNCDk+EpHxwHiATp3s6KaJjNSkOJ664QzGPf4Jv3h+JbCSfpktObdHGjk90xjQMZWgzUM3VYiqRn6jIjOBh1V1uYiMBAap6h+qaTsEmKyq59W23ezsbM3NzY1wtaYpq6hQVm/bx7y1O5m/roAlWwqpUGiZGMuwrDbk9GzLuT3SSGse73epxkMikqeq2bW186qHWQwkureTqWboLyKtgEeAKzyqw5gaBQJC38yW9M1syW3nZVF0sJSFGwqYt7aA+esKeHWFczS9T0YLzjs1nQnndqNZnJ2+3FR59ZvPwxmGfwT0B9ZWbeAe5HkO+KWqbvaoDmPqpGWzWC7p14FL+nWgokL57Ot9zF9XwPy1BTz83npeX/U1074/mO5pyX6Xanzg1ZC8BbAQeBe4ELgauFJVJ1VqczPwO2C5+9A0Vf1PTdu1Ibnx08L1Bdzx72WUlFXw4Nh+XNi3vd8lmQgJd0juSWC6BaQCI4AFqro9Etu0wDR+27b3EBP/tYRlX+3lxmFd+fl3nPU3TXQLNzA9+02raqGqzopUWBrTEHRISWTWhCH8YEhnHlv4Jdc89jE79x32uyxTT+y/RmPqKC4mwG9H9eEvVw9gZX4RFz38AR9v3O13WaYeWGAac4JGDcjgpVvPpkVCDP/z+MfMWPAFXu3iMg2DBaYxJ6FHenNeuvVsLuidzu9eW8PNTy9h3+FSv8syHrHANOYkNU+I5a//M4hJF5/G25/vYNSji8jbXGgsVkzzAAALCklEQVS9zUbIzsA1JgJEhBuGdaNfZgq3PrOEK6Z9SPe0JC4fmMGoARl0bNXM7xJNBHh2WpEX7LQiEw2KDpXy2sqveXFpPp98uQeA7M6pjBqYwSV925OaZAtzNTS+n4fpBQtME222Fh7k5eXbmLM0n3U7iokJCDk90xg9MIPzT0snITbod4kGC0xjGhRV5fOv9zNnWT4vLctnx74jJMfHcEHvdozolc7QrDYkx9seMr9YYBrTQJVXKB9v3M2LS/N5Y9V29h8pIzYonN6lFTk90xjesy2ntE1GxJaXqy8WmMZEgdLyCnI3FTJv3U7mrSlg7Y79AGSkJP43PL91SmtbIcljFpjGRKFtew8xb20B76/dyaINuzhYUk5cMMCZ3VoxLKsNgzql0iejpe37jDALTGOi3JGycqf3uXYn768tYMPOYgBig0Kv9i0Y2CmVQZ1TGdgxhczURBvCnwQLTGMamYL9R1i6pZClX+1lyeZCVmwt4lBpOQBpzeMZ1CnFCdFOqfTLtF5oXfi94roxJsLSmsczsnc7RvZuB0BZeQVrtu9n6ZZClmzZy5Ithby5egfg9EL7ZrTk9C6tyO7SiuzOqXb+ZwRYD9OYRmR38RGWbtnLp5v3kLupkBVb91Ja7nzHT2mbzOldUsnu3IrTu7SiYysbxh9lQ3JjDIdLy1mxtYhPN+0hd9MecjcXsv9wGQBtm8eT3SWVfpkp9MtoSZ/MlrRIiPW5Yn/YkNwYQ0JskDO6tuKMrq0A5yqZ63bu59NNheRu2kPe5kJeW3lsje9ubZKci8JltKR/xxR6d2hhpzRVYj1MY5q4wgMlrMwvYmV+Ecu/2svK/CK+LnJWkQ+IM5Tvm5HCae2bk5XenB7pybRrkdCohvM2JDfGnLCd+w+zKr+IFVuP/ewqPvLf55PjYzilbTJZbZPpkd6cU9Kd2x1aJhIIRF+QWmAaYyJqd/ERNuwsZv3OYtbv2O/8ubOYgv3HgrRZXJDuacl0bt3M+WmV5N5Oom3z+AYbpr7vwxSRmUAvYK6qTq6mTTowW1WHeVWHMSYyWifH0zo5njO7tf7G43sPlvw3SNft2M+GncWszC/ijVXbKas41iGLjwnQqZUTpJ3cIE1vEU9yfCxJ8UGaJ8SQHB9LckIMzWKDDTJcPQlMERkDBFV1iIj8XUSyVHV9lTapwD+BJC9qMMbUj5Rmcc65nl1afePxsvIKtu09zOY9B9i8+yCbdzt/btlzkEUbdv/3pPtQRCApLobk+BiSE2JokRBDrw4tGNQplYGdUunSupkv+1A9GZKLyMPAG6r6mohcDSSq6j+qtGkBCPCSqubUsK3xwHiATp06Dd68eXPE6zXG1C9VpaD4CLuLSyg+Ukbx4TL2HynjQKXbxYfLKD5SSvGRMnYXl7Aqv4gDJU7ItkqKY2DHFAZ2SnFmNnVMOanl8fwekicB+e7tPcCgqg1UdR9Q6/8SqjoDmAHOPsyIVmmM8YWI0LZ5Am2bJ4T9mvIKZf3O/Szd4kwNXfrVXt5dsxNwjub3SG/OoM6pZHdOZcygTE/q9iowi4FE93YydrE1Y8xJCgaEU9u14NR2LfjeGZ0AKDpYyrKtxwL0leXbyNtUGHWBmQcMBT4C+gNrPXofY0wT1rJZLOf2SOPcHmmAc2L+7gMlnr2fVz2/OcA4EZkCXAWsFpGQR8qNMSZSAgEhrXm8Z9v3pIepqvtEJAcYATygqtuB5dW0zfGiBmOMiTTPzsNU1UJgllfbN8aY+mYHY4wxJkwWmMYYEyYLTGOMCZMFpjHGhMkC0xhjwhRVy7uJSAFQl8nkbYBdHpXjF/tM0cE+U3Q4+pk6q2pabY2jKjDrSkRyw5lQH03sM0UH+0zRoa6fyYbkxhgTJgtMY4wJU2MPzBl+F+AB+0zRwT5TdKjTZ2rU+zCNMSaSGnsP0xhjIsYC0xhjwtRoA1NEZorIYhGZ5HctkSAiMSKyRUTmuT99/a7pZIhIuogsdG/HisgrIrJIRK73u7YTVeUzZYjI1kq/r1rP8WtIRKSliLwuIm+JyIsiEhft36lqPlOdvlONMjArX7US6CYiWX7XFAH9gGdVNcf9Wel3QScqxBVDbwPyVPVsYKyINPetuBMU4jOdCdxf6fdV4F91J+QaYIqqjgS2A1cT/d+pqp/pbur4nWqUgQnkcGwtzrdwLpcR7c4CLhGRT9z/6T1by7QelAPfBfa593M49vtaAETjydFVP9NZwA0iskREfudfWSdGVaeq6tvu3TTg+0T5dyrEZyqjjt+pxhqYVa9ame5jLZHyKXC+qp4BxAIX+VzPCVPVfapaVOmhqP99hfhMr+P8R3A6MERE+vlS2EkSkSFAKvAVUf47OqrSZ3qbOn6nGmtgNsarVq5Q1a/d27lANA6JqtMYf18fqup+VS0HlhKFvy8RaQU8AlxPI/kdVflMdf5OReWHDsPRq1aCc9XKTf6VEjFPiUh/EQkCo6nmGklRqjH+vt4UkfYi0gwYCazyu6C6EJE44Dngl6q6mUbwOwrxmer8nYrm/WA1mQMsFJEOwIU4+5Oi3X3AM4AAL6vqOz7XE0n/BF4TkWFAL+Bjn+uJhN8C7wMlwN9UNdouNf0jYBBwj4jcA/wD50qw0fydqvqZ3geeog7fqUY708c9ajkCWOBetdI0YO4XcSjwZpV9gaaBsO9UIw5MY4yJtMa6D9MYYyLOAtMYY8JkgWl8Ja5Qj9dxO10iVZO7vR6hpsqJSLdIvo+JLhaYpl6JyB/dueNBEXkYZwrhwyGa3i8iF4lIkojMEZHmIjKymm3+Aufo58nU1VFE7qj00J1A1xBNbxaRc07mvUz0ssA09S0GGIszC+ZK4FHgMnfxg/srtfs2ME9VDwBdgFLgARHpWHljbs+yo6q+4N7/7wIY7v3jFvaoZrGPfOBCERkpIrFujXdVWphhmtvul8DtImLfnSbIfummvv1KVZ/FmTJ4Ok5PbhowHOdcU0RkBLBNVQ+6rylT1cM4U9e6VNneOOCv7uuqLoABoRf2OO4xVa3AOU9vFHAd8Eec0BwBPATsBlDVMuBd4OyI/G2YqGKBaeqNiPwImCciY4FLgCdxepg/wJnX+46I9AfuB0pE5EwRuRXoKCKzgT8AZ1TZbHdV/dy9XXUBDAi9sEeox1DVfFW9BRjo1vW/OMHYAXi+0jY/ctuYJqaxzvQxDZCqzhSRI0Cyqg4FEJF/AvtU9Tb3/kSchSt6AhnAWpyFR36hql/Usv197jYqPxxqYY+Qi324q9U8pqrXuffbAvGqWvW6L4c4Nq/aNCHWwzS+EZEzgbZAQaUl0B7DWRwBVX3BXY7rVeCCajZzSESSa3ibUItGVLeQxA+AL9zakoHuwOUhttkVZ/Ue08RYYBpfuPspZwITVPU+4BQR+amqloZo/jzwQxEJhDjd6DXgihreKtSiEcc9JiLtgfHAg+7jDwK/Bg66uxIqG42zJqRpYmxIbuqNe2T5Upyh9jvAFaq6xX36BuC8o03dHwBUtUBEXgH+AuwQkbWq+pz79KvAf0TkdVXdGeJtQy3skR/isR8Dvwc6icifgPmq+rKIvA7MEpGzgEk4S4AVqOquSPydmOhic8lNvRGRCTjD2XeAW4BOOKcLxQIJQArOUfPFwKOqOrrK63+Ds/L3Oaq6rdLjme5jz1Tzvsct7FHdYh8i8gMgv/LKNW6vdiLwEnAV8Eg1PWHTyFlgmnojIgH39B1jopIFpjHGhMkO+hhjTJgsMI0xJkwWmMYYEyYLTGOMCZMFpjHGhOn/A14G+DaSmBDYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "parameters = L_layer_model(train_x, train_y, [train_x.shape[0],20, 7, 5, 1], num_iterations = 2500, print_cost = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率:0.9856459330143539\n"
     ]
    }
   ],
   "source": [
    "pred_train=predict(train_x,train_y,parameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率:0.8\n"
     ]
    }
   ],
   "source": [
    "pred_test=predict(test_x,test_y,parameters)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
